{"cells":[{"cell_type":"markdown","metadata":{"id":"zfqERVVKxIU0"},"source":["# Implement RNN unfolding using RNNCell and LSTMCell\n","\n","<br>\n","<font color='#2c3e50'>\n","    \n","- RNN, LSTM and GRU cells can be seen as neural networks that process each step of the input time-series (in somewhat recursive manner). This is often referred to as `unrolling or unfolding of RNN`.\n","- LSTM/RNN/GRU layers provide an abstraction over this operation.\n","</font>\n","\n","## 1. Task:\n","\n","<font color='red'>\n","\n","<b>In this lab exercise, you have to implement `unfolding or unrolling` using `RNNCell` and `LSTMCell`. </b> \n","</font>\n","\n","\n","<font color='#2c3e50'>  \n","<ul>    \n","<li>\n","\n","RNNCell (or LSTMCell) takes a feature vector and hidden state (or hidden and cell state) at a time-step and outputs the updated hidden state (or hidden and cell states). The hidden state obtained after the last time-step is considered as a latent representation of the time-series and is given as input to the dense or classification layer(s). </li>   \n","    \n","<li>\n","    \n","This notebook contains two PyTorch model definitions (i.e. `LSTMClassifier` and `RNNClassifier`) and `trainer()` function to process the training data and train models. The entire code is in `almost` working condition. You need to complete the `forward()` function in both `LSTMClassifier` and `RNNClassifier` classes. In essence, you need to implement RNN unrolling in `forward()`. </li>   \n","\n","    \n","<li>\n","    \n","If you are not comfortable with PyTorch, please see the Reference notebook. It summarises how to define and train a model in modular manner. </li>\n","    \n","\n","<li> \n","    \n","Since RNN cells are essentially composed of standard dense or linear layers, you are welcome to implement your own RNNCell instead of using the implemented one.  \n","    \n","    \n","</ul>\n","</font>\n","    \n"," \n","### 1.1 HINTS:\n","<br>\n","<font color='#2c3e50'>  \n","<ul>    \n","<li>\n","    \n","Run a loop over time-steps </li>\n","\n","<li>\n","    \n","Neural network process one batch at a time. So, input to `RNNCell` at each time-step is `(batch_size,n_features)`</li>    \n","</ul>\n","</font>\n","\n","### 1.2 Instructions:\n","\n","- You have to implement `forward functions` in both `LSTMClassifier` and `RNNClassifier` classes. \n","- These functions take a batch of time-series of size `(batch_size,n_steps,n_features)` and output mortality predictions (1/0 binary predictions) of shape `(batch_size,1)`.\n","\n","\n","### 1.3 Data:\n","\n","- Physionet 2012: Mortality prediction (discussed yesterday)\n","- Each time-series consists of K-dimenional vector (`n_features=K`) at 48 time-steps (`n_steps=48`). `Size of each time-series : (48,K)`\n"]},{"cell_type":"markdown","metadata":{"id":"hdc1v7fvxIU5"},"source":["## 2. Synatx of RNNCell and LSTMCell\n","\n","\n","### 2.1 RNNCell:\n","\n","\n","#### PyTorch syntax:\n","<img src=\"./img/rnn_cell_1.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/rnn_cell_2.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/rnn_cell_3.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/rnn_cell_4.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/rnn_cell_5.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","\n","### 2.2 LSTMCell\n","\n","#### PyTorch syntax:\n","<img src=\"./img/lstm_cell_1.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/lstm_cell_2.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/lstm_cell_3.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","<img src=\"./img/lstm_cell_4.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n","\n","\n","\n","### 2.3 More information at:\n","\n","- http://pytorch.org/docs/stable/generated/torch.nn.LSTMCell.html\n","- http://pytorch.org/docs/stable/generated/torch.nn.RNNCell.html"]},{"cell_type":"markdown","metadata":{"id":"sw9jhQgQxIU6"},"source":["## Implementation Begins:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"8Ust8VKMxIU7"},"outputs":[],"source":["import torch\n","from torch import nn\n","from torch.nn import functional as F\n","from tqdm import tqdm\n","import numpy as np\n","from sklearn.metrics import roc_auc_score\n","from utils import get_validation_score\n","import warnings\n","warnings.filterwarnings('ignore')\n","import matplotlib.pyplot as plt"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"f39vXbYTxIU9"},"outputs":[],"source":["class RNNClassifier(nn.Module):\n","    \n","    def __init__(self, input_dim, hidden_dim,device):\n","        super().__init__()\n","        self.hidden_dim = hidden_dim\n","        self.rnn = nn.RNNCell(input_dim, hidden_dim)  # RNN Cell\n","        self.fc = nn.Linear(hidden_dim, 1)            # fully connected layer: maps last hidden vector to model prediction\n","        self.activation = nn.Sigmoid()                # coz binary classification\n","        self.device=device\n","    \n","    def forward(self, x):\n","\n","        hidden = self.init_hidden(x)\n","        \n","        ############################# \n","        \n","        # Write you code here.\n","        # Return expects variable out. Its the hidden vector obtained after last time-step.\n","        \n","\n","\n","\n","\n","        ###########################\n","        \n","        return self.activation(out)\n","    \n","    def init_hidden(self, x):\n","        h0 = torch.zeros(x.size(0), self.hidden_dim)\n","        return h0.to(self.device)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ikvKS54axIU-"},"outputs":[],"source":["class LSTMClassifier(nn.Module):\n","    \n","    def __init__(self, input_dim, hidden_dim,device):\n","        super().__init__()\n","        self.hidden_dim = hidden_dim\n","        self.rnn = nn.LSTMCell(input_dim, hidden_dim)\n","        self.fc = nn.Linear(hidden_dim, 1)\n","        self.activation = nn.Sigmoid()\n","        self.device=device\n","    \n","    def forward(self, x):\n","        hidden,cell = self.init_hidden(x)\n","        \n","        ############################\n","        \n","        # Write you code here.\n","        # Resturn expects variable out. Its the hidden vector obtained after last time-step.\n","        \n","\n","\n","\n","        #############################\n","        \n","        return self.activation(out)\n","    \n","    def init_hidden(self, x):\n","        h0 = torch.zeros(x.size(0), self.hidden_dim)\n","        c0 = torch.zeros(x.size(0), self.hidden_dim)\n","        return h0.to(self.device),c0.to(self.device)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Pdk6nqo8xIU-"},"outputs":[],"source":["# Function to compute validation score: Used in trainer()\n","\n","def get_validation_score(model,Val_T,Val_L):\n","    model.eval()\n","    tensor_x = torch.Tensor(Val_T).to(model.device)\n","    preds=model(tensor_x)[:,0]\n","    LOSS=nn.BCELoss().to(device)\n","    val_loss=LOSS(preds,torch.Tensor(Val_L).type(torch.FloatTensor).to(model.device))\n","    return roc_auc_score(Val_L, preds.cpu().detach().numpy()), val_loss"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"qjKFBLTyxIU_"},"outputs":[],"source":["def trainer(model,training_set,validation_set,device,lr,stored_name,epochs=10):\n","    \n","    # Recieves data and labels \n","    T,L=training_set \n","    Val_T,Val_L=validation_set\n","    \n","    # intialise optimiser and criterion\n","    \n","    optimizer_model = torch.optim.SGD(model.parameters(),lr,momentum=0.9, nesterov=True)\n","    criterion = nn.BCELoss().to(device)\n","\n","    # \n","    best=0\n","    LOSS=[]\n","    VAL_LOSS=[]\n","    \n","    # training begins\n","    \n","    for epoch in range(0,epochs):\n","        Loss=0\n","        model.train()\n","        for k in range(0,len(T)):\n","            \n","            inputs=T[k]\n","            labels=L[k]\n","            \n","            inputs=torch.Tensor(inputs).to(device)\n","            labels=torch.Tensor(labels).type(torch.FloatTensor).to(device)\n","            \n","            pred=model(inputs)\n","            \n","            loss=criterion(pred[:,0],labels)\n","            optimizer_model.zero_grad()\n","            loss.backward()\n","            optimizer_model.step()\n","            Loss=Loss+loss\n","           \n","        Val_ROC,val_loss=get_validation_score(model,Val_T,Val_L)\n","        VAL_LOSS.append(val_loss.detach().cpu().numpy())\n","        LOSS.append((Loss/len(T)).detach().cpu().numpy())\n","        \n","        print(' Epoch: {:.1f} Training Loss {:5f} Validation Loss {:.4f} Validation AUC {:.5f}'.format(epoch,LOSS[-1],VAL_LOSS[-1],Val_ROC))\n","        \n","        # If current validation score is greater than best, store the model\n","        \n","        if best<Val_ROC:\n","           torch.save(model, './'+stored_name) \n","\n","    return torch.load('./'+stored_name).to(device),LOSS,VAL_LOSS     "]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":845,"status":"ok","timestamp":1636212380550,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"8itcyfT3xIVA","outputId":"a8415f54-8b80-4f65-c715-a0d00320c399"},"outputs":[{"name":"stdout","output_type":"stream","text":["(33, 48, 59)\n"]}],"source":["# Get training and validation data\n","\n","from get_data import get_training_data,get_validation_data\n","\n","T,L=get_training_data(batch_size=32)  # returns lists of training data and label batches\n","Val_T,Val_L=get_validation_data()     # numpy arrays of validation data and labels\n","\n","print(T[0].shape)                     # (batch_size,time_steps,n_features)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"y43nY3esxIVC"},"outputs":[],"source":["device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"wrOne0ySxIVC"},"outputs":[],"source":["n_features=T[0].shape[2]     # 76 dimensional vector at each time step\n","recurrent_units=128          # number of hidden units in  a RNN/LSTM\n","lr=0.001                     # learning rate "]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"executionInfo":{"elapsed":13,"status":"ok","timestamp":1636212380553,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"NnxeESThxIVD","outputId":"69d9927a-96aa-49ef-e904-e980717c0396"},"outputs":[{"name":"stdout","output_type":"stream","text":["LSTMClassifier(\n","  (rnn): LSTMCell(59, 128)\n","  (fc): Linear(in_features=128, out_features=1, bias=True)\n","  (activation): Sigmoid()\n",")\n"]}],"source":["# Create LSTMClassifier model object\n","\n","model=LSTMClassifier(n_features,recurrent_units,device)\n","print(model)"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":82127,"status":"ok","timestamp":1636212462671,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"dGsbdDO9xIVD","outputId":"9910184d-d6d3-4703-bda4-dfc8e44f0eda"},"outputs":[{"name":"stdout","output_type":"stream","text":[" Epoch: 0.0 Training Loss 0.540809 Validation Loss 0.4646 Validation AUC 0.66463\n"," Epoch: 1.0 Training Loss 0.442982 Validation Loss 0.4385 Validation AUC 0.71209\n"," Epoch: 2.0 Training Loss 0.426729 Validation Loss 0.4273 Validation AUC 0.74553\n"," Epoch: 3.0 Training Loss 0.414635 Validation Loss 0.4157 Validation AUC 0.76728\n"," Epoch: 4.0 Training Loss 0.401894 Validation Loss 0.4036 Validation AUC 0.78027\n"," Epoch: 5.0 Training Loss 0.388968 Validation Loss 0.3919 Validation AUC 0.78843\n"," Epoch: 6.0 Training Loss 0.376999 Validation Loss 0.3819 Validation AUC 0.79447\n"," Epoch: 7.0 Training Loss 0.367230 Validation Loss 0.3745 Validation AUC 0.79893\n"," Epoch: 8.0 Training Loss 0.360319 Validation Loss 0.3698 Validation AUC 0.80231\n"," Epoch: 9.0 Training Loss 0.355960 Validation Loss 0.3672 Validation AUC 0.80499\n"]}],"source":["# Train LSTMClassifier\n","\n","model=model.to(device)\n","model, training_loss, validation_loss=trainer(model,(T,L),(Val_T,Val_L),device,lr,stored_name='lstm',epochs=10)"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":391},"executionInfo":{"elapsed":404,"status":"ok","timestamp":1636212463051,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"dNz56XhfxIVE","outputId":"96fea02f-60c8-47db-afe0-0571394183d8"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYAAAAF2CAYAAABnDE+7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gVRdvA4d+THlJIIIHQEoqQUFRKFBBE9AVEKRaKShFRXmyoYKNGymsBAf3sgIJSlaYCFlCwoSAIKKg06QEEEiCFkj7fH3sSQwwpcHLOSfLc17VXTnZnZ2cM7nN2ZnZGjDEopZQqf9ycXQCllFLOoQFAKaXKKQ0ASilVTmkAUEqpckoDgFJKlVMaAJRSqpzSAKCUUuWUQwOAiLiLyGQRiRORZBFZKiIhF0nbXkSMiJzJta3Lk8aIyLk8aSrmOl5BRGaJSIJtmykiviVdT6WUKg0c/QQwArgNaAnUtO2bW0D6TGOMf67tunzSdMqTJjHXsdeAKCASaAA0BF65/GoopVTp5+gAMBiYZIzZZ7tRPwt0FpEIe1/I9k2/HxBjjDlujDkBxAADRMTH3tdTSqnSxsNRFxKRICAc2Jy9zxizV0SSgKuBg/mc5i4isYCn7bxRxpitedIsFhFPYC9WcPnYtj8S8Ml9PWAL4Iv1NLAtT/kGYwUo/Pz8WkRGRl5SPZ0lKysLADe38tWto/XWepcHl1PvLVu2xBtjQvM75rAAAATYfibm2Z8ABOaTfifQFPgT8AeGA9+IyJXGmKO2NB2An2yfbwPmi8gdxpiVF7le9ud/Xc8YMwOYARAdHW02bdpU1Hq5hNTUVAC8vb2dXBLH0nprvcuDy6m3iOT35RpwbBNQsu1nxTz7g4CkvImNMceMMVuNMRnGmARjzEjgFHBLrjRrjDEptm0hMA/oW8D1sj//63pKKVXeOCwAGGMSgENA8+x9IlIX69v4toudl0cWIEU8vgtIyX09oBlwHthdxOsppVSZ5eiGtBnAcBGpIyKBwCRglTHmQN6EInKTiFwhIm4i4i8i44CqwCrb8SYicq2IeImIp4jcDvQHFgEYY85jPRFMEJEqIlIFmADMMcakOKCuSinl0hwdACYCK4BfgCOAO9ZIHUSkr4icyZX2amANVlPOPqAV0NEYE2s7Hgq8D5wGTgBjgPuNMctz5TEU69t+9rYLGFYiNVNKqVJGdEGYf9NO4NKjPNc7KyuLuLg4zp496+ziOEz2/UqkoJbgsqewevv5+VGzZs18RwmJyGZjTHR+5zlyFJBSyo5Onz6NiBAZGVluhkXqMNB/1zsrK4sjR44QHx9PlSpVipVv+fqvqFQZkpycTNWqVcvdzVBdyM3NjapVq5KYmHeEfRHOLYHyKKUcIDMzE09PT2cXQ7kAT09PMjIyin2eBgClSrHy1hau8nep/w40ACilVDmlAUAp5bJuueUWZs+ebfe0yqKjgOwkPTWDw3+donL1AAIr6ZIDqvzy9/fP+Xzu3Dm8vb1xd3cHYPr06fTt2/dip/7Ll19+WSJpi+O7776jX79+HD58uETydyZ9ArCTD8Z9x8T7PuX3tRedd0mpcuHMmTM5W3h4OCtWrMj5PffN/1I6LZV9aQCwk/CG1myrB3fEO7kkSrmm7777jpo1azJp0iTCwsIYOHAgp0+fpmvXroSGhhIcHEzXrl0v+Kbdvn173nvvPQA++OAD2rVrxzPPPENwcDB16tS54Ft/3rRt27bl6aefzjft/v37adeuHQEBAXTo0IFHH32Ufv36FbtOO3bsoH379gQFBdG4cWOWL/9nIoIvvviCRo0aERAQQI0aNZgyZQoA8fHxdO3alaCgICpVqsT111+fM87f0bQJyE5qN7IFgO1xTi6JKq8ejJ7hkOtM3zT4ks89duwYp06d4uDBg2RlZXHu3DkGDhzIokWLyMzM5P7772fIkCF8+umn+Z6/YcMG+vfvT3x8PDNmzOCBBx7gyJEj+Y6C2bBhAwMGDMg3bZ8+fWjTpg2rV69m48aN3HrrrXTv3r1YdUlPT6dbt27cf//9fPXVV/z444/cdtttbNq0icjISB544AEWLVrE9ddfz+nTp9m/fz8AU6dOpWbNmsTFWfeKn3/+2WmjufQJwE7Co6yljQ//dZKM9Ewnl0Yp1+Tm5sb48ePx9vbG19eXypUr06NHDypUqEBAQACjR4/m+++/v+j5ERER/Pe//8Xd3Z0BAwbw999/c/z48WKlPXToEL/88gsTJkzAy8uLtm3bFvvmD9aN+8yZM4wYMQIvLy9uuukmunbtyocffghYY/O3b99OUlISwcHBNG/ePGf/33//zcGDB/H09OT66693WgDQJwA78fX3omp4RY4fSuTo3tM5AUEpR7mcb+aOEhoaio/PPyuynjt3jmHDhrFy5UpOnz4NWG84Z2Zm5nQc5xYWFpbzuUKFCoDV55Cfi6WNj4+nUqVKOfsAatWqRWxs7L/yKMjRo0epVavWBW9iR0REcOTIEQCWLl3K888/z4gRI7jqqquYOHEirVu35plnnmHcuHF06tQJgMGDBzNixIhiXdte9AnAjiJszUAHtBlIqXzl/aY7depUdu3axYYNG0hKSuKHH34A/pn8rCRUq1aNU6dOce7cuZx9xb35A1SvXp3Y2NgL2u8PHTpEjRo1ALjmmmtYtmwZJ06c4Pbbb6d3794ABAQEMHXqVPbt28fy5ct55ZVXWLNmzWXW6tJoALCj7ABwcIcGAKWKIjk5GV9fX4KCgjh16hTjx48v8WtGREQQHR3NuHHjSEtLY/369axYsaLQ81JSUi7Yrr32WipUqMDLL79Meno63333HStWrODuu+8mLS2N+fPnk5iYiKenJ4GBgTlPCp999hl79uzBGEPFihVxd3d32nxOGgDsKKcj+E8NAEoVxdChQzl//jwhISG0atWKzp07O+S68+fPZ/369VSuXJkxY8Zw1113FTil+JEjR/D19b1gi42NZcWKFXz55ZeEhITwyCOPMGfOHKKiogCYO3cutWvXJjAwkGnTpjF//nwA/vrrLzp06IC/vz+tW7fmkUce4cYbb3RIvfPS9QDycanrAaScS2do+w8Qgde+H4iXj+O6WMrzvPhQPuu9d+9eGjVq5OyiOFRJTQd91113ERUV5ZAnkEtRlHrv2LGDhg0b/mt/QesB6BOAHflU8KRa7SCyMg1H9pxydnGUUhfxyy+/sHfvXrKysli5ciXLli3j9ttvd3axHE4DgJ1pR7BSru/YsWO0b98ef39/Hn/8cd555x2aNWvm7GI5nA4DtbOIhiGs/2y3dgQr5cK6detGt27dnF0Mp9MnADuL0DeClVKlhAYAO6tZvzJu7sLf+xNIPZ/u7OIopdRFaQCwMy8fD6rXq4TJMsTuOuns4iil1EVpACgBEQ2taSC0H0Ap5co0AJSAiIbaD6CUcn0aAEpA7ca6NoBSyvVpACgB1etVwsPTjeMHEzh/Js3ZxVGq1BAR9uzZA8BDDz3E//73vyKlLa758+fnzMZZEg4cOICIuPyqZxoASoCnlzs16lfGGDi0U58CVPnSuXNnnnvuuX/tX7ZsGWFhYUW+KU6bNo2YmJjLLk9+N+O+ffvy1VdfXXbepZ0GgBKiHcGqvBowYADz5s3715TOc+fOpW/fvnh46PunrkIDQAnRjmDlcCKO2Qpx++23c/LkSdauXZuz7/Tp03z22Wd0796d1q1bExQURLVq1RgyZAhpafk3k953332MGTMm5/fJkydTo0YNatasyaxZsy5I+/nnn9OsWTMCAwOpVasW48aNyznWrl07AIKCgvD392f9+vU5awZnW7duHddccw0VK1bkmmuuYd26dTnH2rdvT0xMDG3atCEgIIBOnToRH1+8J/ujR4/SvXt3KlWqxBVXXMG7776bc2zjxo1ER0cTGBhI1apVefLJJwFr+ul+/fpRuXJlKlWqRMuWLS+6+tmlcmgAEBF3EZksInEikiwiS0Uk36WzRKS9iBgROZNrW5freAMRWSIiR2x5/Skig/Lk8Z2IpObJo2tJ1xNyrw2gTUCqfPH19aV3797MmTMnZ9+iRYuIiorC39+fV199lfj4eNavX8+aNWt4++23C81z5cqVTJkyhVWrVrFr1y5Wr159wXE/Pz/mzJlDQkICn3/+Oe+8807OusLZi8wkJCRw5swZWrdufcG5p06dokuXLjz++OOcPHmSJ598ki5dunDy5D/v8SxYsID333+fEydOkJaWlrPAe1Hdfffd1KxZk6NHj7JkyRJGjRrFN998A8ATTzzBE088QVJSEnv37s1ZOGb27NkkJiYSGxtLXFwcb7/9Nr6+vsW6bmEc/QQwArgNaAnUtO2bW0D6TGOMf67tulzHgoFvgWuAQOBBYIqI3Jknj//lyeMz+1SlYNXrBuPp7U7c4STOJqU64pKqvDPGMVsRDBgwgCVLlpCSkgLAnDlzGDBgAC1atKBVq1Z4eHhQu3ZtHnzwwQLXAM62aNEiBg4cSJMmTfDz87vgGz5Y39KvvPJK3NzcuOqqq7jnnnuKlC9YTw/169enf//+eHh4cM899xAVFXXBIjEDBw6kQYMGOcHtt99+K1LeYK029tNPPzFp0iR8fHxo2rQpgwYNygmQnp6e7Nmzh/j4ePz9/WnVqlXO/pMnT7Jnzx7c3d1p0aIFgYGBRb5uUTg6AAwGJhlj9hljEoFngc4iElHcjIwxG4wxbxljjhrLj8BKoL19i3xp3D3cqNmgMgCHtB9AlTNt27YlJCSETz/9lL1797Jx40b69OnD7t276dq1K2FhYQQGBjJq1KgiNadkr7+bLSLiwlvGhg0buPHGGwkNDaVixYpMmzatyM00R48e/Vd+udf2hX+vL3yxdYgvln+lSpUICAjIN/+ZM2eye/duoqKiuOaaa/jsM+s7av/+/bn55ptznh6GDx9Oerp9p5dxWG+MiAQB4cDm7H3GmL0ikgRcDRzM5zR3EYkFPG3njTLGbL1I/hWAVkDecWNDReRJ4G9gHjDFGFPgf0VjTM5CI5ejVoNK7P/9BHt/P0bdpqGXnV9B7FHe0qg819sYc8F6tK6mf//+zJ49m507d9KpUydCQ0Pp06cPTZs2Zf78+QQEBPDaa6+xdOnSC+qRlZVFVlYWxpicOoaFhXHo0KGcjuUDBw5ckLZPnz48+uijfP755/j4+DBs2DDi4+Nz8smdNvtz9s+wsDAOHjz4r7V9O3XqdEG6/M69mLz5nzp1isTExJwgcPDgQapXr05WVhb16tVj/vz5ZGVl8fHHH9OzZ0/i4uLw8/MjJiaGmJgY9u/fT7du3WjQoAEPPPBAvte8lPuWI58AssNfYp79CVhNOHntBJoCdYAoYBvwjYhUz5tQRNyxmpJigTm5Do0E6gOhwAPAIGBCfoUTkcEisklENsXF2ecbe60o2xPATp0TSJU//fv3Z82aNbz33nvce++9gLUGcGBgIP7+/uzcuZNp06YVKa9evXoxe/Zstm/fzrlz55gw4cL/jZOTkwkODsbHx4eNGzfy4Ycf5hwLDQ3Fzc2Nffv25Zv3rbfeyu7du1mwYAEZGRksXLiQ7du307WrfboLa9WqxXXXXceoUaNISUlh27ZtzJo1i759+wIwb9484uLicHNzIygoCLBW/vr222/5/fffyczMJDAwEE9PT/uvHZwdZUt6A4IAAzTNsz8R6F7EPP4CHsizzxNYBKwDKhZyfl8gtrDrtGjRwtjDkT0nzeAW083IrvPtkl9BUlJSTEpKSolfx9WU53r/+eefzi5GoW644QYTFBSU8zf6/vvvTWRkpPHz8zNt27Y1MTExpk2bNjnpAfPXX38ZY4wZMGCAGT16dM6xl156yVStWtVUq1bNzJw584K0ixcvNuHh4cbf39906dLFPProo6Zv374558bExJiQkBBTsWJFs379evP+++9fcN21a9ea5s2bm8DAQNO8eXOzdu3aC+rw7rvv5vye99z87N+/3wAmPT3dGGNMbGys6dKliwkODjZ169Y177zzTk7avn37mtDQUOPn52caNWpkPvnkE2OMMQsWLDANGjQwFSpUMFWqVDFDhgzJyS8/27dvz3c/sMlc5F7n0DWBReQgMN4YM8v2e11gL1DHGHOgCOfvAiYbY96z/e4DLAH8ga7GmAIb5kTkHtv5NQtKd6lrAueVlZnF0PYfkHo+gylf9ycg2L49+LmV57VxoXzWW9cELj/KyprAM4DhIlJHRAKBScCq/G7+InKTiFwhIm4i4i8i44CqwCrbcX/gS8ALuCXvzV9EgkSkq+1cEZFmwDhgYQnW7wJu7m7Uisp+IUyHgyqlXIujA8BEYAXwC3AEcAf6AYhIXxHJfRO/GlgDJAP7sDp4OxpjYm3He2CN+GkLxOUa55/dqOgJjLFdJwnrxr8Aq1/AYfSFMKXKpvnz5+Pv7/+vrXHjxs4uWpE59J1sY0wm8LRty3tsPjA/1++vAq8WkNdsYHYBx+OwgoZT1dYlIpUqk/r27ZvTkVtala+GNCfQOYFUSXJkH55yXZf670ADQAkLrVURHz9PEuLOkRB31tnFUWWIt7c3J0+e1CBQzhljOHnyJD4+PsU+V6flK2FubkJEw1B2bTrKwR3xBIX6ObtIqowICwvjxIkT2Ou9ldIgO9hJESalK0sKq7ePjw81axY4uDFfGgAcIKJhiC0AxHF1u2LPeqFUvjw8PKhTp46zi+FQ5XnYL9i/3toE5AAR2hGslHJBGgAcIGco6I54ba9VSrkMDQAOEFIjgAqB3iSfOs/p49oRrJRyDRoAHEBEdDioUsrlaABwEH0hTCnlajQAOEh2R/ABDQBKKRehAcBBtCNYKeVqNAA4SHBVPwIq+XIuKZX4I8nOLo5SSmkAcBTtCFZKuRoNAA6kL4QppVyJBgAHyt0PoJRSzqYBwIFyNwFlZWlHsFLKuTQAOFBQqB9BoRVIOZtOXGyis4ujlCrnNAA4mDYDKaVchQYAB9MXwpRSrkIDgIPpUFCllKvQAOBg2U8AsTvjycrMcnJplFLlmQYABwsI9qVyNX9Sz2dw7ECCs4ujlCrHNAA4gXYEK6VcgQYAJ9COYKWUK9AA4ATaEayUcgUaAJwg3NYEdHj3STIztCNYKeUcGgCcwC/Qm9CagaSnZnJ032lnF0cpVU5pAHCSnGYg7QdQSjmJBgAnyZkaWvsBlFJO4tAAICLuIjJZROJEJFlElopIyEXSthcRIyJncm3r8qS5QkRWi8hZETksIk/lOV5BRGaJSIJtmykiviVZx6LSoaBKKWdz9BPACOA2oCVQ07ZvbgHpM40x/rm267IPiIg7sALYAYQC3YHhInJXrvNfA6KASKAB0BB4xV6VuRzhUVbcO7z7JOlpmU4ujVKqPHJ0ABgMTDLG7DPGJALPAp1FJOIS8moHRAAjjTHnjDFbgOnAQwC2b/r9gBhjzHFjzAkgBhggIj72qMzl8PX3ompERTIzsji695Szi6OUKoc8HHUhEQkCwoHN2fuMMXtFJAm4GjiYz2nuIhILeNrOG2WM2Wo7djWw2xhzJlf6LcCjts+RgE/u69mO+2I9DWy7WFmNMaSmphajdpemVlRljh9MZM/WvwmrG3hZeTmivK5I612+aL3ty5FPAAG2n3lXQkkA8rv77QSaAnWwmnG2Ad+ISPVc+RWUV37Xy/78r+uJyGAR2SQim+LiHNMxGx5VGYBDO0865HpKKZWbw54AgGTbz4p59gcBSXkTG2OOAcdsvyYAI0WkJ3ALMNOWX0F55b5eQq7PXOR6M4AZANHR0cbb27vwGl2meldWA+Dw7lPY63qOKLcr0nqXL1pv+3DYE4AxJgE4BDTP3icidbG+jV+0OSaPLEBsn7cCDUTEL9fxZrb9ALuAlNzXsx0/D+wubvlLQq3IyoibcHTvKdJSMpxdHKVUOePoTuAZWCN16ohIIDAJWGWMOZA3oYjcZBvm6SYi/iIyDqgKrLIl+QGr3+BFEfEVkabAg1gdwRhjzgPzgAkiUkVEqgATgDnGmJSSrWbRePt6Uq1OEFmZhsN/aTOQUsqxHB0AJmIN3fwFOAK4Y43UQUT6ikjuDt2rgTVYTTn7gFZAR2NMLIAxJhPoBjQBTgJfAJONMR/lymMo1rf97G0XMKykKncpcl4I0zeClVIO5sg+gOyb9tO2Le+x+cD8XL+/CrxaSH57gP8UcPwscL9tc0kRDUNZv2K3vhCmlHI4nQrCyWrrE4BSykk0ADhZjSsq4eYu/H0ggZRz6c4ujlKqHNEA4GRePh7UqFcJk2U4vFs7gpVSjqMBwAXoEpFKKWfQAOACdCSQUsoZNAC4AO0IVko5gwYAF1C9XjAenm4cP5TI+TNpzi6OUqqc0ADgAjw83alZP3tiOH0fQCnlGBoAXIR2BCulHE0DgIvIWSRe1whWSjmIBgAXoSOBlFKOpgHARVSrE4yntzvxR5I5m+gSk5Uqpco4DQAuwt3DjVqR2c1A2hGslCp5GgBciPYDKKUcSQOAC4loqP0ASinH0QDgQnI6grUJSCnlABoAXEhYREW8K3hy6tgZkk6dd3ZxlFJlnAYAF+Lm7kZ4lK0fQJuBlFIlTAOAi9GOYKWUo2gAcDHaEayUchQNAC5GO4KVUo6iAcDFhNYMxNffi8T4cyTEnXV2cZRSZZgGABfj5iaEN9SOYKVUydMA4IJy+gG0GUgpVYI0ALggXSJSKeUIGgBcUO6hoMYYJ5dGKVVWaQBwQZWrB+BX0Zvk0ymcPq4dwUqpkqEBwAWJyD9LRP55wsmlUUqVVRoAXJR2BCulSppDA4CIuIvIZBGJE5FkEVkqIiFFOO9hETEiMibXvutF5EyeLUNEtuVK84GIpOdJ80hJ1c+etCNYKVXSHP0EMAK4DWgJ1LTtm1vQCSISATwF/J57vzFmrTHGP3sDAoEjwLw8WczOnc4Y87Y9KlLStCNYKVXSHB0ABgOTjDH7jDGJwLNAZ9tN/mJmAqOBU4XkfSsQBrxvl5I6WVAVPwIr+3IuOY34I8nOLo5SqgzycNSFRCQICAc2Z+8zxuwVkSTgauBgPuc8CJw1xiwUkYcLucRDwFJjTN42kx4icicQDywDxhtjzhSUkTGG1NTUQutU0mpFVubPdYfZs/UogaHeBaZ1hfI6g9a7fNF625cjnwACbD8T8+xPwGq+uYCIhANjgELb7G1pbwGm5zn0BhAFhAB3ADcA714kj8EisklENsXFuUa7e62oygAc2nnSySVRSpVFDnsCALLbMSrm2R8EJOWT/j3geWPMkSLkPQjYZYz5PvdOY8zmXL/+KSLDgO9E5D5jTGqetDOAGQDR0dHG27vgb9yOUO/KMGArh3efoqjlcYVyO4PWu3zRetuHw54AjDEJwCGgefY+EamL9e1/Wz6ndAReFJF4EYkH2gAjRWRt7kQi4gE8wL+//ecnK/u04tfA8bKHgh7aGU9WlnYEK6Xsy9GdwDOA4SJSR0QCgUnAKmPMgXzS1sLqG2hq2zYBbwE986TrBgQDs/NmICJ32/oeEJH6wFRguTEmxT7VKVkVQyoQVMWPlLPpnDiUt+VMKaUuj6MDwERgBfAL1pBNd6AfgIj0FZGczlljzOHcG5AKJBljjufJ80Fgoe0JI6+HgH0ichb4CvgZGGjvSpUkXSJSKVVSHNkHgDEmE3jatuU9Nh+YX8C57S+yv3NxzylNIhqGsvX7gxzYHkfLW+o7uzhKqTJEp4JwcbUb6xvBSqmSoQHAxWV3BMfuOklmRlYhqZVSqug0ALg4/yAfKlcPIC0lg2MH8uvmUEqpS6MBoBTQjmClVEnQAFAK5EwNrf0ASik70gBQCmQvDqNrAyil7EkDQCkQHmU1AcXu1o5gpZT9aAAoBfwCvalSK5CMtEyO7i1sVmyllCoaDQD2kpUFS5ZACS3eoktEKqXsTQOAvfTtC716wRtvlEj2EbpEpFLKzjQA2EtP2xx1Tz0F69fbPfvsoaAHdCioUspONADYS48eMGwYZGRA794Qb9+mmvCoEETgyF+nSE/LtGveSqnySQOAPU2aBK1bw+HD0K+f1S9gJz5+XoTVDiIzI4sjf+kKYUqpy3dJAUBEfEWkQyGLuZc/np6wcCGEhMCqVfDCC3bNXjuClVL2VKQAICIfiMgjts9ewEas+fV3icgtJVi+0qdWLZg/H0Rg7FhYvdpuWWtHsFLKnor6BHAz1mIqAN2xFngPA8bZNpVbp07w3HPWkNA+feBIUZY1LpzOCaSUsqeiBoBg4ITtc2dgqTHmBPAR0KgkClbqxcRAx44QFwd33QXp6ZedZa3IEMRNOLrvNGkpGXYopFKqPCtqADgGNBERd6yngex2DX/g8u9sZZG7u9UUVKMG/PQTjBx52Vl6+XhQvW4wWZmG2N3aEayUujxFDQCzgIXAH0AmsMa2vyWwswTKVTaEhsKiReDhAVOnwscfX3aWOc1A2g+glLpMRQoAxpgJwP3ADKCtMSbNdigDmFRCZSsbrrsOXn7Z+jxwIOzZc1nZ/TMzqAYApdTlKfKi8MaYpfnsm23f4pRRQ4fCjz9aTwC9esG6deDre0lZ6VBQpZS9FHUYaG8R6ZTr9+dE5LCIrBKRaiVXvDJCBGbNgiuugN9+g8cfv+SsatavhJu7cGz/aVLOafeLUurSFbUPYFz2BxFpDowCXgc8gan2L1YZVLGiNVuojw+89x7MvrSHJ09vD2pcUQlj4NBOfQpQSl26ogaACGCX7fMdwKfGmJeBJ4H/lETByqSrr4a33rI+P/ww/P77JWVTW18IU0rZQVEDQArWy19g3fCzh4Em5tqviuL++63O4PPnrQnkkpKKnYV2BCul7KGoAWAtMFVEYoBo4Avb/gZAbEkUrEx780248kr46y8YNKjYi8hoR7BSyh6KGgCGAGlAT+AhY8xR2/5bgFUlUbAyrUIFqz8gIAAWLy72IjLV6wXj4eXOiUOJnEtOLaFCKqXKuqK+B3DYGNPNGHO1MWZWrv1DjTGXPqSlPGvQwBoZBPD00/DzzwWnz8XD052a9SsB2hGslLp0xZoOWkRuEpEhIvKoiNxYUoUqN3r2hCeesOYJKuYiMjozqFLqchX1PYAaIrIR+BoYDowAVovIBhGpXvuAgxAAACAASURBVNSLiYi7iEwWkTgRSRaRpSISUoTzHhYRIyJj8uw3InJORM7k2irmOl5BRGaJSIJtmykil/YGVkl5+WVo1QpiY4u1iIz2AyilLldRnwBex5oD6ApjTC1jTC2gvm3f68W43gjgNqw5hGra9s0t6ATbojNPARcbM9nJGOOfa0vMdew1IAqIxOqwbgi8UozyljwvL2u+oMqVrUVkXnyxSKfV1pFASqnLVNQA0BF41BizP3uHMWYf8LjtWFENBiYZY/bZbtTPAp0LWVlsJjAaOFWM62D7pt8PiDHGHLdNXx0DDBARn+LkVeJyLyLz3HNFWkQmrHYQnt7uxB9J5kxCigMKqZQqa4o8FxCQ31jFIo9fFJEgIBzYnHOyMXtFJAm4GjiYzzkPAmeNMQtF5OGLZL1YRDyBvVjBJXvKzUjAJ/f1gC2AL9bTwLaLldUYQ2qqg0fXtG+P+8iReLz4IqZPH9J+/tmaSroANRtUYv/vcezZdpS6TQttSSuTHP53chFa7/KlpOpd1CeANcAbIlIre4eIhAP/xz9TQxcm+4WxxDz7E4DAvIlt+Y8BHikgzw5AHazmpFeA+SLSuYDrZX/O73qDRWSTiGyKi3NOs0rm6NFk3XQTEheHZ79+hS4iEx5l3fQP7dS1AZRSxVfUJ4DHgeXAPhHJfgegOla7fJ8i5pFs+1kxz/4gIL/XYd8DnjfGXHQ9RWNM7uCzUEQ6AH2BlXmul5Dn2v+6njFmBtZ010RHRxtvb++L16QkffQRNGuG2/r1eI8fD5MnXzRp3SZV+X7xDo7sPk12eZ1WbifTepcvWm/7KOp7ALFAc+BWYIptu8UY09x2rCh5JACHbPkAICJ1sb6N59cc0xF4UUTiRSQeaAOMFJG1BVwmCxDb511YU1g0z3W8GXAe2F2UMjtFaCgsXGitKDZlCnzyyUWT1m5cBdCOYKXUpSnyewDG8rUx5g3btlpEGovIicLPzjEDGC4idUQkEGsxmVXGmAP5pK2F1TfQ1LZtAt7CehsZEWkiIteKiJeIeIrI7UB/YJGtvOeBecAEEakiIlWACcAcY4xr95q2afPPIjL33Qd79+abrEp4RbwreHL6+FmSTp53XPmUUmVCsV4Ey4cHULkY6ScCK4BfgCOAO9ZIHUSkr4icyU5oe/s4ZwNSgSRjzHFbklDgfeA01oL1Y4D7jTHLc11vKNa3/extFzCs2LV0hmHD4I47rMnieva0Jo/Lw81NiLD1A8Tu0n4ApVTxXG4AKBZjTKYx5mljTIgxJsAYc6cxJt52bL4xxr+Ac9sbY57P9fu3xpjGxhg/Y0ywMSbaGPNRnnPOGmPuN8YE2bYHbE8Grk8E3n8f6tWzFpF54ol8k2W/EaxTQiilisuhAUAVU/YiMt7e8O67+S4ik71IvI4EUkoVlwYAV9e0aYGLyGQ/AcTuPIkp5rTSSqnyrcAAYJuvJ+liG7DOQeUs3+6/HwYMsPoBevaE5OScQ6E1A/H19yLp5HkS4885sZBKqdKmsPcAhjikFKpgIvD227Bli/UEMGiQ9b6ACCJCRKNQdm48wqEdJ6las5KzS6uUKiUKDADGmEtbuVzZX/YiMtHR1uRxbdvCY48BUKdJFXZuPMInb/xC9TqVqXGFBgGlVOG0D6A0adAAZs60Pj/1FGzYAECHPlcSHlWZk0fPMGngp/z67f4CMlFKKYsGgNKmVy94/HFrnqBeveDkSfyDfHj87c606FiH1PMZTHvmaz57dzNZWdoprJS6OA0ApdHkydCy5QWLyHh5e3Dv2Ou587FrEYEV0zfz7ojVpJwreEI5pVT5pQGgNMq9iMzKlTmLyIgINw9oyqOvdsbHz5Mt3+xn8gPLiD+aXEiGSqnySANAaRUeDvPmWSOExo5Fvvkm59CVbcMZOfsOqoZX5PBfp3jp3k/YvfloAZkppcqjIgcAEXlTRHR4iSvp3BnGjIGsLDwHDED++ivnUFjtIEbMvp3G19XiTEIKrz7yOd8v2e7EwiqlXE1hL4LVzPVrH8Dftv/33IvDKCcaOxb+8x9rEZnmzeHppyHBWv6gQoA3Q169mY79ryIr07Bg4o/Mf3EtGemZTi60UsoVFPYEsFNEDorIAqzlFbNv+rUBz5IsmCoid3dYvJjMvn2R9HSYOhWuuMKaPiI9HTd3N3o+0YqB49vj4eXODx/v4NVHPifpVOmYE08pVXIKCwBBQC+sdXXdgC9EZDfgDdwsIlVLuHyqKIKDyZg5k7R166BdOzh5EoYMgauugs8/B2No1aUBT8/oRlBoBfb8eoyX7v2E2F06g6hS5VlhAcDTGLPRGDMVayWtZsBAIBO4H9gvIrtKuIyqiEzz5vDdd/Dxx9Y00jt3Qteu0KkTbNtGnSZVGDnnDuo0qcKpY2d4+YHlbF69z9nFVko5SWEBIEFENojIK4AX4GuM+QnIAO4CgoEHSriMqjhErIVktm+HV16BoCBYvRqaNYP//pegzGSemt6V1l0bkJaSwYwRq1n2zi/60phS5VBhAaAG8DzWalwewGbbmrxeWGvtGmPMjyVbRHVJvLysVcX27LHeHHZzg/feg/r18ZwyiQHPXkuvYa0QN+GLmb8y7ZmvSDmb5uxSK6UcqMAAYIyJN8asMMaMBM4B1wBvAAZrYfhEEfm+5IupLlnlyvDaa/DHH9C9O5w5A2PGIFFRdJA/ePz/bqZCgBdbvz/IpIHLiDuc5OwSK6UcpLgvgiUaYxYB6cBNQB3gbbuXStlfZCQsWwZr1sDVV1vTSPTtS6NhvYgZUo1qdYI4uu80Lw34hB0bjzi7tEopByhOALgKOGz7fBBIN8YcM8YstH+xVIm56SbYvBlmzYKwMNi4kUo9OzMmcxFtrvTkbGIqrz/2Bd989IeuMKZUGVfkAGCMiTXGZNk+NzHGxJZcsVSJcneHgQPhr7/guefA1xePT5bSf8FDPBW8Fq+0cyycso65z/9Aepq+NKZUWaVzAZVn/v4wfjzs2gX9+yNpaTRYPY8p+8Zz06kfWP/pdl596DNdalKpMkoDgIJatWDOHPjlF7j+ejyTTnPX/vmM2/08PmvX8NK9H3NwR5yzS6mUsjMNAOof0dHw/fewdCnUrUvVM0d5fM8b9P/pJebf8zYbV+5xdgmVUnakAUBdSATuvNN6kWzqVEzFijRO2s6IreNJ6X8/X7z4JVmZWc4upVLKDjQAqPx5e8OTTyJ79mCGDAE3N9rFr+XGmDv5ud0gzsfr+wJKlXYaAFTBQkKQN97A7c8/SL6+I75ZKVy37n1SIq4g4c1ZoENFlSq1NACooomKIuCHr0j4cBnHKkYQfC6OoMce4NxV0bB+vbNLp5S6BBoAVLEE3d2doEM7+LbjUBI9Aqnwxxa47jpMnz5w+HDhGSilXIZDA4CIuIvIZBGJE5FkEVkqIiFFOO9hETEiMibXvgYiskREjtjy+lNEBuU57zsRSRWRM7m2riVRt/LEJ9CXG1a+wo+Tl/N52K2kiSfy4YeYBpHw/PNwXhebUao0cPQTwAjgNqAlkL3c5NyCThCRCOAp4Pc8h4KBb7EmqAsEHgSmiMidedL9zxjjn2v77DLroAA3N6HL0BsIm/sGL17zApuCWyDnz0FMDKZhI2tNAu0fUMqlOToADAYmGWP2GWMSgWeBzrab/MXMBEYDp3LvNMZsMMa8ZYw5aiw/AiuB9iVUdpWPFh3qMuzzR9n26MtMbfAksb41kYMHoEcP6NABfs8bt5VSrsLDURcSkSAgHGt5SQCMMXtFJAm4GmuCubznPAicNcYsFJGHC8m/AtAK+F+eQ0NF5Engb2AeMMUYk15QXsYYUlNTi1Ar1+HM8voEuNN3TBv+urUeMye3oP7mz7ntyDL8v/kG07QpWYMHk/Hcc1Cpkt2vXdr+Tvai9S5fSqrejnwCCLD9TMyzPwGrCecCIhIOjAEeKSxjEXHHakqKBebkOjQSqA+EYq1cNgiYcJE8BovIJhHZFBen0x5civrNw3hmzu0EjX2S/7V4kW9D22OyDO7TpuHVpAlu06dDRoazi6mUshFHTflrewI4DTQzxvyWa38i0N8YszxP+q+ApcaY6bbfvwNWG2Oez5POE5iP1adwi61p6WJl6AtMNMbUKqis0dHRZtOmTcWpntNlf0Pw9vZ2ckksp46dYdHUdRz/4ifuil1IVLJt6eirrrIWqGnf3i7XcbV6O4rWW+tdVCKy2RgTnd8xhz0BGGMSgENYS0lmF6wu1rf/bfmc0hF4UUTiRSQeaAOMtC1JmX2+D/AJUAXoVNDN3yYLkMuqiCqSSmH+PDS5Ez1mPMDc9s8xre6DxHtVhm3b4MYboVcvOPivVj+llAM5uhN4BjBcROqISCAwCVhljDmQT9paWH0DTW3bJuAtoCeAiPgDX2KtT3yLMeZM7pNFJEhEuoqIv1iaAeMAXcDGgZq0CWfsot7UeHYQzzf9H8uqdyfVzQuWLMFERcHYsXBOp5tWyhkcHQAmAiuAX4AjgDvQD6zmGRHJuYkbYw7n3rAWpk8yxhy3JemBNeKnLRCXa5z/NNtxT6w+hCNAEtaNfwFWv4ByIC8fD7o9GM2oJX042OMhxjYez8bga5CUFJgwAaKiYOFCHTaqlIM5rA+gNNE+gJJjjOHXbw+waOo6Ku3dyt2xHxF+zra4XLt2Vv9A06ZFzq+01NvetN5a76JyiT4ApQBEhOY31WHc4t7Ue/hOJjYezdzwfpzxDIAffsC0aAEPPQTx8c4uqlJlngYA5RQ+FTzp8UQrxnzYixM392ZMowmsrvIfsozA9OlQvz68/jqkF/jKhlLqMmgAUE5VvV4lnpzelbtf7MLKqwcwoWEM2wMbQUICPPGE1Ry0erWzi6lUmaQBQDmdiNDq1vpMWNqbqAEdeL3BE7xV7xHiK1SxVibr2BFuvx327XN2UZUqUzQAKJdRIcCbe55tw6i5d5LUpgNjI5/j4xp3kObhA8uWQcOGMGoUnDlTeGZKqUJpAFAuJzwqhOHv385dY25ibYPbGNNwPBtCWkFaGrz0EkRGwrx5OmxUqcukAUC5JDc3od2dDZmwtDdNerZkVsRAJkYOJzaoLhw9Cv37Q5s2yObNhWemlMqXBgDl0gKCfbn3uRt4dmZ30ptF80LdZ5gdcS/nKgTD+vV4tm2Lx8MPw8mTzi6qUqWOBgBVKtS7OoxRc++k51Nt2BLRnpENxvJ19Zsxbu64v/++1Sw0cyZkZTm7qEqVGhoAVKnh7uFGhz5XMn5Jb668pQlLqt3JuKgY9oU2tp4ABg2Ctm3ht98Kz0wppQFAlT5BoX4MevE/DH27C1n1GzCp1mO8V+cBzgdUhvXroUULGDoUkpKcXVSlXJoGAFVqNby2BiNmd6fTgKvYHNqSEfVi+LH2zRiw5hSKjIQPP9TRQkpdhAYAVap5ervT9cHmjJ53J2FXhjO38p08HzmK49UbwrFj0KePtTbxzp3OLqpSLkcDgCoTatavzPD3b6PXk605UakOY8Me56PIgaQHVIRvvrFWIhs1StceUCoXDQCqzHBztzqJxy7qRcPW4Xzr34rhdZ/j94adrEnlXnoJGjWC5csLz0ypckADgCpzQqoH8PgbtzBwwo1QuTJvVujB1CtHkhQeaS1Dedtt0K0b7N/v7KIq5VQaAFSZlD3B3Pglvbn2livY7VWb4SFPsKrF/WT5B8Bnn1lPAy+8ALbFNpQqbzQAqDItINiXB/53E4+91pmgahX5mJaMvCKGQy06QUoKjBlj9Q98/bWzi6qUw2kAUOWCtTh9L266pwmJHhV5gR7Mum40qbWvgN27oVMnuPtuOHLE2UVVymE0AKhyw6eCJ3c9dR3D37+d6vWC2ZAazrDgYWzp8F+Mr6+1MH1UFLz6KmRkOLu4SpU4DQCq3KnTpAqj593JbQ9HI95eTD8dzYstXuD0dR2ttQaefNJ6m/inn5xdVKVKlAYAVS55eLpz6wPNifmwJ1c0C+PQeT9GpPbks+7jyKwVAdu2WfMK3X8/xMU5u7hKlQgNAKpcC6sdxFPTu9F3ZFt8/DxZcaQaw8NHcqDHQxgvL8ieaXT6dJ1pVJU5GgBUuefmJrTr0Yhxi3vTtH1tklOElw40471ur5Latj2cPg0PPQStW4MuQKPKEA0AStkEV/Hj4SmdePDljgRW9mXTAQ+eTO/D5sETMdWrw8aNcM01MGQIJCQ4u7hKXTYNAErl0fymOoxb3Ju2d0SRkWGYsTmYide+REL/B8HNDd56y2oWmjtXZxpVpZoGAKXy4RfoTf/R7XhyWleqhFfkQGwKI3Y058vHZ5HV+jo4cQLuvRduuAG2bnV2cZW6JBoAlCpAZHR1Yhb0oPPApoib8OkPKYwOeojYkVMgNBTWroXmzeGRR3RdYlXqaABQqhBePh7c8ei1jJp7JxGNQjl14jzPfxXAnH7vkjr4ERCBd96BBg1g2jTIzHR2kZUqEocGABFxF5HJIhInIskislREQopw3sMiYkRkTJ79V4jIahE5KyKHReSpPMcriMgsEUmwbTNFxNfe9VLlQ60GlRk+6zZ6Dm2Fl48HP/1wnJF7rmXr68swN94Ip07Bww9DdDT8+KOzi6tUoRz9BDACuA1oCdS07Ztb0AkiEgE8BfyeZ787sALYAYQC3YHhInJXrmSvAVFAJNAAaAi8ctm1UOWWu4cbHftdxdiFPWnYsgZnE1N5e9YR3mz8NMnvzoXwcGtR+uuvh379dG4h5dIcHQAGA5OMMfuMMYnAs0Bn203+YmYCo4FTefa3AyKAkcaYc8aYLcB04CEA2zf9fkCMMea4MeYEEAMMEBEfu9ZKlTshNQJ54s1bGTD2BioEePHH+sOMnpPODy8sxcTEgLc3zJ9vjRaaOFGnnFYuycNRFxKRICAcyHmTxhizV0SSgKuBg/mc8yBw1hizUEQeznP4amC3MeZMrn1bgEdtnyMBn9zXsx33xXoa2HaxshpjSC1l/8OWtvLai7Pr3aJTba5oXoXFUzew7YdDzP+/Lfx8dXP6fbGWqq89j/vy5TByJFkzZ5I5ZQpZnTvb5brOrrezaL3ty5FPAAG2n4l59icAgXkTi0g4MAZ4pID8Csorv+tlf87veoNFZJOIbIrTuV9UMVQMqcCgl27k/ufbE1DJh71bT/DCmK182T2GlGXLyYqMxG3PHjxvvx2PO+6AvXudXWSlAAc+AQDJtp8V8+wPApLySf8e8Lwx5mKNqMmF5JX7egm5PpPf9YwxM4AZANHR0cbb2/sil3VtpbXcl8sV6t2ycwOatA5n8as/s/6z3ax4Zwtbo0K4d+l31Fq1AMaNw/3LL3Ffs8aacXT0aPD3v6xrukK9nUHrbR8OewIwxiQAh4Dm2ftEpC7Wt/H8mmM6Ai+KSLyIxANtgJEistZ2fCvQQET8cp3TzLYfYBeQkvt6tuPngd2XXyOl/s2vog/3jWvP42/cQqUwfw7tjOfF+1fwqc/1pP++He67D9LSrH6ByEhYsEDfJlZO4+hO4BlYI3XqiEggMAlYZYw5kE/aWljt/E1t2ybgLaCn7fgPWP0GL4qIr4g0BR7E6gjGGHMemAdMEJEqIlIFmADMMcaklFQFlQJo3LoWYxf2pH3vxpgsw5ezfuX5p9axd+hLsH69NVT06FHo21ffJlZO4+gAMBFr6OYvwBHAHWukDiLSV0RyOnSNMYdzb0AqkGSMOW47ngl0A5oAJ4EvgMnGmI9yXW8o1rf97G0XMKxkq6iUxcfPi3uebcPT73anakRFjh1IYPKg5Xy0NpOUb3+E997Tt4mVU4nRx89/iY6ONps2bXJ2MYole5RAeWsbLS31Tk/N4PP3trBqzlayMg2Vq/nTb3Q7GkX5w7hx8Oab1hvElSrB88/D4MHg7n7R/EpLve1N6138eovIZmNMdH7HdCoIpRzA09uD2x+9llFz7qBWZGVO/n2G14Z8wQev/cbZCZOsJqCbbrLeJn7kEWtJyrVrC89YqcugAUApB6oVGcLI2Xdwx5Br8fByZ/2K3YzrtYgtxyvA6tWwZIn1NvHWrdCundVHoG8TqxKiAUApB3P3cKPzfU2JWdCDK5qGkXTyPNOf/Zrpw1eTeMMtsGMHPPec9TbxggX6NrEqMRoAlHKSsNpBPDWjG/cMb4N3BU+2fLOfcb0Xs/6bw5hx46xAcMcdcPYsjBwJTZrA5587u9iqDNEAoJQTubkJ7Xs1ZuzCnjS+rhbnklL5YNx3vP7Yl8R7h8DHH8NXX0HDhrBnD3TtCl27Inv2OLvoqgzQAKCUC6hcLYDHXuvMwPHt8avozfafDzPhrsV889EfZP2ng9Un8MorEBgIn3+OZ/PmuA8fbq1MptQl0gCglIsQEVp1acC4Rb1o0aEuqeczWDhlHVP+u5xjR87CsGGwaxfcdx+SlobHa69BnTrw7LOg81epS6ABQCkXE1i5AoMnduChyR0JrOzL3q3H+d89S/hi1q9khlSB998nbf16Mrt0gXPnYPJkKxCMHKkvkqli0QCglItqdmMdxi3uTZvbIslIz2LZ27/w0r2fcGhnPKZZMzKWLoWNG+HWW62O4okToXZtGDPGep9AqUJoAFDKhfkFenNvzA0MfetWKlcPIHb3SV4a8AnL39lMWmoGXHONNTLo55/h5pvhzBl44QXriWDsWEhIKPwiqtzSAKBUKdCwZU3GLuzJf+5pgskyrJ73BxP7L+ePnw5ZCVq2hJUrYd066NgRkpJgwgTriWD8eEjMu3SGUhoAlCo1vH096f3UdTwz8zaq1Qki/kgybzyxknee/or4o7blL1q3toaNrl1rTS2RmGjNNVS7tjXHUFJ+S2+o8koDgFKlTL2rqvLsB924/bFovCt48tt3BxjXaxFfzNxCelqmlahtW1izBr7/3ppuOiEBYmKspqGXXoLk5IIvosoFDQBKlULuHm7cdHdjJiztzTU31yM9NZNl72xiwl2L+XN97D8J27WD776Db76B66+3OodHjbICwaRJVp+BKrc0AChVigWF+jHohf/w5LSuVKsTxInYJF5/7EumPfMVp47lurnfeKP1NPD113DdddZw0REjoG5dmDLFGk6qyh0NAEqVAZHR1RmzoAc9nmiJt68Hv357gOd6LOTL93/9p1lIBDp0gB9/hFWrrI7juDh45hnrieDVV+H8eedWRDmUBgClyggPT3c69b+a8Ut6E92xLumpmXz61i9MuHsJ238+/E9CEejUyVqa8osvrKGkJ05YC9XXrQuvvw4pumpqeaABQKkyJriqP/99qQND3+5CWO0gThxK5LUhXzD92a8vbBYSgVtugQ0bYMUKa1nKY8fgiSegXj1rlTINBGWaBgClyqiG19Yg5sMe3PnYtXj7erDlm/2M7bmIlR/8RkZ65j8JRaxZRjdtgmXLoGlTa8H6xx6D+vXhnXd0LYIySgOAUmWYh6c7Nw9oyrjFvWnRoS5pKRl88uZGJty9hB0bDl+YWAS6d4ctW6xpqK+6Cg4ftpaorF8fZsyAtDTnVESVCA0ASpUDlcL8GTyxA0+8eStVwyty/GAi//foF8wYsZrTx/MMBRWxFqL59VdYvBgaN4bYWHjwQWjQAN57D9LTnVMRZVcaAJQqRxq1qknMRz25Y8i1ePl4sHn1Psb2XMSq2XmahQDc3KBnT9i2DRYuhEaN4OBB+O9/rXWLn3kG/vjDORVRdiHGGGeXweVER0ebTZs2ObsYxZJqa6P19vZ2ckkcS+t96fU+dewMi19Zz5Zv9gPWEpV3P9uGhtfWyP+EzExYtMiaUmL79n/2N28OAwZAnz4QEnLJ5SkK/XsXv94istkYE53vMQ0A/6YBoPTQel9+vf9cH8tHk9dx4pA1YVx0x7r0HNaa4Cp++Z9gjDVy6IMP4KOP/ploztMTunSB++6zRhd5eV122fLSv7cGgBKnAaD00Hrbp97paZl8PW+bNZ9Qaibevh50+W8L/nNPEzw83S9+YkoKLF9uBYNVqyAry9ofEmI9EQwYAM2aWf0KdqB/bw0AJU4DQOmh9bZvvU/+ncyiV9bz27cHAKhWJ4h7hrclMrp64Sf//TfMn28Fgz///Gf/lVdagaBvXwgLu6zy6d9bA0CJ0wBQemi9S6bef/x0iIVT1nEi1po++pqb69FzaCuCQi/SLJSbMdZQ0tmzYcGCf5apdHeHzp2tYNCtG/j4FLtc+vfWAFDiNACUHlrvkqt3emqG1Sw061erWaiCJ90Gt+Cmu5vg7lHEAYRpadaKZbNnWz8zMqz9QUFwzz1WMLj22iI3EenfWwNAidMAUHpovUu+3vFHk1n8ynp+++4AANXqBtP7ydY0bFkDKU7b/okT8OGHVjD49dd/9kdFWYGgXz+oWbPALPTvbd8A4ND3AETEXUQmi0iciCSLyFIRyXfcmIhcLyJbROSUiCTaPt+Z5/iZPFuGiGzLleYDEUnPk+YRR9RVqbIipHoAD0/pxJD/60xozUD+3nea14Z8wcsPLOePdbEU+UtklSrWPENbtsDWrdbkc1WqwM6dMHKk9W5Bp05Ws5FOT+0QDn0CEJHRwACgM3ASmAVUMMbckk/aKoAvYFv0lOuBlUALY8yOfNK7AfuBt4wxL9v2fQBkGGMGFaec+gRQemi9HVvv9NQM1nz4B1/N3crZRKsMEY1C6TKoOVddH168JwKw3ihetcp6Kli+/J+pJgICoHdva0hpmzY5TUT69y7FTUAichCYYIyZafu9HrAHqG2MOVjAeW7AdcBqoJcxZkU+aboCS4Gaxpg4274P0ABQpmm9nVPvlHPpfL9kO1/P20byKWsNgVoNKnProOY0bV8bN7dLGPZ56pT1XsHs2bBx4z/769WDe++Fe+8ltVo1QP/exeESAUBEgoDTQDNjzG+59icC/Y0xyy9yXgLgB3gAP/D/7d15dNX1mcfx9ycLJBAISwnKYiCAyI4GBEHUIgVsZ+rCtJWxjstMaaenM7XTzkzPmXiJNwAAEq9JREFUaXvO2Jm24+h06plOZ1S0tmpP944bI2BdWhGUTSGiBhIIqyEISdgx5Jk/vr/AzTUJSUjuzc19Xufcw81v/T5Afs/9fb+/+31gvpl9aGpCSc8AtWb25zHLHgWuBww4ADwJ3G1mLdbBKy4utldffbVtASZZsi8IyeJxJzfuUyfqWPVkKb9/ooTa90MiuLCoHwtum8zUjxaSkdm+Xma98w4Zjz9O5s9+hvbuPbO8bs4cPrj+ejKvvhobPz48WZQGzuffOycnp0skgOGE7pwiM9ses7wC+IaZPd7Cvj2B64CxwH1mdjpu/UWE7p+5ZvZyzPJiYDdQBYwDfgyUmdniJs6xBFgCMHz48OKtW7e2N9Sk6CoXhETzuLtG3KdO1rHm6a08/0QJ1ftD//3gwnzm3zaZy64d0fqnhuKdPo1eeIHMxx4j46mnUEx9AsvPp37mTGzWLOpnzcKmTYPc3I4Ip8vpDgmgXXcAccdYBjxpZg/ELf828GdmNv4c+88GXgLymrqLaOBdQKnD4+5acX9w6jSrn36X5x59g/f3hRvtguF9ue7OS5lx3Zj2JwKAmho++PnPyXjpJTJXrw4T08XKzobiYrjyyvCaPbvT5yZKlJTvAooaUkHognkk+rkIKANGmtmOVuy/AthkZl+LWZYFVAD/Zmb3n2P/KwjdSH3MrNlSR54AUofH3TXjPl1Xz5pnS1n2yEYO7DkMwEeG9mHh7VO54k8ubnl6iRY0invXLli1KtQ4fuWVMGtp/PXskkvOJoQrrwwlLztoWopE6i4J4BvAX3D2KaCHCRfjhU1suwgoBd4m9P/fCvw3cJ2ZrYzZ7kbgCWCImVXHHeNm4Dkzq5Y0BvgJsM/MFrXUTk8AqcPj7tpxn66rZ+3ybSx7eCOV0WRz/Qf3ZuHtU5n9ybFk98xq0/FajLumBtasOZsQXnvtw0XuL7igcUKYMgWy2taGZOguCSATuAe4HegJrASWmNkBSbcAD5hZXrTtl4AvAxcCp4B3ge+b2a/ijvkc4aJ+RxPnewmYHJ1rP/A74J/MrLaldnoCSB0ed2rEXX+6nnUry1n2yEb2lR8CoN+gXiy4bSpX3nAJPXJadxFuU9ynToUvnDUkhFdegQMHGm/TuzfMnHk2IcycCXl5bYotEbpFAkgVngBSh8edWnHX1xsbX9jOsoc3sHvrQQD6Dsxl/q1TuGrROHrmZre4/3nFbQalpSERNHQdxT/skZkZaiLHjiNEj54mkyeABPIEkDo87tSMu77e2PSHCp5duoGd74RP5Xn9cvjYZydzzafGk9O76VoCHR53ZWXjcYQNG0Lhm1ijRp1NBsXF4ef8/I45fyt5AkggTwCpw+NO7bjNjJJVu3h26Qa2l+wHoHd+T65dPIm5N08kN69xIuj0uI8eDWMHDQlh9Wo40sTXhgYMCAPKTb2GD+/wcQVPAAnkCSB1eNzdI24z4+3X9vDMQ+spe7MSgNy8HsxdPJFrb55I7/wwdXTC466rg82bzyaELVugvLzluYoyM6Gw8GxCGDWqcYLo16/NzfAEkECeAFKHx9294jYzStfv45mH1lO6fh8AOb2zuebTE/jYLZPJzg2PcCY1brPQdVRe3vRrz56W9+/fv+W7h+wPj4N4AkggTwCpw+PuvnFv3biPZ5du4O3XwgW1Z24Ws28Yy0dvHk/B0P5Jbl0LTpyAHTuaTxBHjza/b2ZmmBU1LjGcGjYMGzmSnu0YkPYE0EaeAFKHx9394y7bVMmypRsoeXUXAMoQl0wbwrT5o7h07kh6902hvwMzqKpqPjns3v3hL7PF2rkz3CW0gSeANvIEkDo87vSJe8eWKpY9vJ6SVbs5XReKz2dlZzDhiuFMXzCKyVcVnvMx0i7v5MkwxUVcYqgvK0N79qDKyjZPgOcJoI08AaQOjzv94j5ae5Itq/by+vJtlK7be+YDc4+cLKZcXcj0BaOZcMWwdk830RWdPHkSzOjZjjrKngDayBNA6vC40zvumgPHWP98Oa8v38b2zfvPbNerTw8unTuS6QtGM7b4wnZPS91V+CBwAnkCSB0et8fd4MCeWtauKGPt8jL2bDt4ZnnfgblM+9gopi8YxciJBW2vWtYFeAJIIE8AqcPj9ribsrfsIK8vL2Pt8m1nZiOFMCPptPmjuHzBaIaOHtCpbe1IngASyBNA6vC4Pe6WmBkVW6p4fXkZ61eWUV119gtcQ4r6h2SwcDSDhvXtlPZ2FE8ACeQJIHV43B53a9WfrmfrxvdYu3wbG17YfqaoPcCI8YOYvmAU0+aPot+g3h3W3o7iCSCBPAGkDo/b426Pug9O8/Zre1i7fBtvvFzByWMfAKFWzJjLLmT6/FFcdm0Ref3a/tRNZ/AEkECeAFKHx+1xn69TJ+rY/MpOXl++jZJVu6g7FWYDzcgU42cOY/qC0Uy9urDZGUoTobMSQNcvheOcc52oR04WxfOKKJ5XxPEjp9j44nbWrijjndf3ULJqFyWrdpHdM5PJcwqZes0IiiYVMHBIn5R8miie3wE0we8AUofH7XF3ltqDx9nw+3LWLi9j2xvvNVrXZ0AuRRMLGDmpgKJJgykcP4icXp33LWTvAkogTwCpw+P2uBPh4HtHWLeyjNIN+9i+eT9Hqk80Wq8MMXT0gEZJoeCifDIyOuYuwRNAAnkCSB0et8edaGZG1e5ayjfvZ/vmSspL9rO79H3qTze+lvbq04MREwuipDCYkRML2j1xnY8BOOdcFyCJguH5FAzPZ+bHxwBhIHnnOwco31zJ9s37Kd9cSXXVMbas3s2W1bvP7Du4MJ+iyYPPJIUhRf3JzEreNBWeAJxz7jz1yMli9NQLGD31gjPLDlUeCXcJJSEhVLx9gMqKGioralj9dCkQahwUjh9EUXSHUDSpgL4DeyWs3Z4AnHOuE/QfnEfx4DyK5xUB4bsHu0vfb5QUDuw5TOn6fWeqnwEMHNKn0VjC8LEDO62NngCccy4BsrIzGTGhgBETCs4sqz14nO0lZ8cSdrxVxft7D/P+3sOsXVEW9uuRybCLBzBiwiA+cedlHXqH4AnAOeeSpO+AXKZcVciUqwqBMF3Fvu3VZ8cSSvazr/wQO0qqqHirihv++vIOPb8nAOec6yIyMjMYOnoAQ0cPYM6N4wA4dvgkpRv3ULWrlty8jv02sicA55zrwnr16cm4GUMZN2Nohx87tcvkOOecazdPAM45l6Y8ATjnXJpKaAKQlCnpXklVkg5L+o2kjzSz7RxJGyQdlFQTvb8pbhuTdEzSkZhXfsz6XpIekVQdvR6WlNvZcTrnXCpI9B3A14HrgRnAsGjZY81s+y5wIzAQ6AfcBTwuaVzcdvPNLC/mVROz7n7gEmAscDEwDvh+h0TinHMpLtEJYAlwj5mVRxfqfwAWSiqM39DM9ptZhYXZ6gTUE9o7ujUnij7pfxb4lplVmtl+4FvAbZK6Rpkf55xLooQ9BiqpH3ARsL5hmZmVSaoFpgAVzexXDfQmtPUPwIq4TX4lKRsoIySX30bLxwI5secDNgC5hLuBTc211czOzL6XKlKtvR3F404vHnfHSuQdQJ/oz5q45dVA3+Z2MrN+QB6hO2gZUBezeh4wktCd9H3gCUkLWzhfw/sPnU/SEknrJK2rqqo6dzTOOZfiEvlFsMPRn/lxy/sBtS3taGYngf+VtIyQMB6Ilv8+ZrNfSJoH3AI8F3e+6rhzf+h8ZvYg8CCEegCpOs96qrb7fHnc6cXj7hgJuwMws2pgJ3BZwzJJRYRP4812x8TJAsa0sL6eMF4AYRD5ROz5gEuB40BpK8/nnHPdVqIHgR8E/lHSSEl9gXuA5Wa2I35DSYskTZKUJSlH0ueAucDyaP1ESZdL6iEpW9INwK3ALwHM7DjwOPBtSQWSCoBvAz81sxPx53POuXST6ATwr8DTwFpgD5BJeFIHSbdIOhKz7YXAbwndN3uBO4HFZrYyWj8I+DFwCNgPfBO408yeijnGXYRP+w2vd4GvdEpkzjmXYrwmcBMkVdHMU0ld3EeAA8luRBJ43OnF426bQjMb1NQKTwDdiKR1zRV/7s487vTicXccnwvIOefSlCcA55xLU54AupcHk92AJPG404vH3UF8DMA559KU3wE451ya8gTgnHNpyhOAc86lKU8AKUzSPZLeklQraa+khyQNSHa7EklShqRXo+pww869R/cgaZ6kNVEVvAOSfpTsNnU2SRdI+kVUUfCQpBckTUl2uzqSpJsl/TH6na5rYv3C6Hf+uKQSSfPP53yeAFLbacJUGgMJNRWGAY8ms0FJ8BXgWLIbkUiSrgF+DdxH+LcfBixNZpsS5EfAAEI9j8HAOuAZSWpxr9RyiBDnXfEroskzfwt8jzCz8feA30ka0d6T+VNA3UhUC+GXZtZsfYXuRNLFwP8Bi4CNwHAz253cVnU+SauBl83s68luSyJJ2gT8MJq6HUljgXeAQWbWraaGiJL882aWFbPsbmCumc2JWfbHaLu723MevwPoXq4F3kx2IxJBUgbwCPA1ztZ76PYk9QYuB7IkbYi6f16SlA5TI9wLLJI0KCrrugR4pbtd/FswhcYVDiFUOWx3N5gngG5C0iLgC8CXk92WBPky8J6Z/S7ZDUmw/oTf28XA7cAQQpnUZVHZ1e5sFWEG4f3AEeAm4HNJbVFi9aGNFRXPxRNANyDpU8BDwCfNbEOy29PZJI0Gvgp8KdltSYKGSnc/NrNNZnaK0BecDcxKXrM6V3TH9zxhWvd8oBfwHeCPkgYns20JdJh2VFRsiSeAFCfpDkKJzD81sxeT3Z4EuZJQD6JE0gHCbTDAJklfTF6zOp+Z1QA7gPjBO2tiWXcygFD/+z/NrNbMTpnZUsI17IrkNi1h3qRxhUMIVQ7b3e3rCSCFSfpbwpMgC8xsVbLbk0C/BEYBU6PXx6Pl84GfJqtRCfQj4A5J4yVlAX8PnAReTW6zOk/Uz18KfFFS76hS4J2EbpHWlpTt8iRlRuMbPaKfc6KXCP+3p0laHFVBXAwUAz9p7/kSWRTedbz7gTrgxdgn4cwsL2ktSgAzO0bMo5/RRRDCmMCRpvfqVu4jXPheAHIIT0BdF90ddGc3EAaCKwhdXtuAT5lZeVJb1bFuJVQ6bHA8+nOkmZVJugn4d8IDEOXAjU2V1G0tfwzUOefSlHcBOedcmvIE4JxzacoTgHPOpSlPAM45l6Y8ATjnXJryBOCcc2nKE4BzKS6aDO6HyW6HSz2eAJwDohkmT0XfMs2WdFTSRefY5/aoEE1Tr5xEtd259vJvAjsXXAG8aWZHJc0ADprZzlbsd4wwLUUjZnaioxvoXEfzOwDnglmE6YYhTDbX2rmVzMzei381rIy6Z/5H0v1RGcNDku6NZrds2Ka/pJ9E645Lel7ShNiTSJoZlUA8Kqkmej8kZpMMSd+N6gPsl3Rf3DlukrQpOv5BSS+n0SyarhmeAFzaknSRpGpJ1cDfAZ+P3n8XuCFa1xG1dm/h7KyVnycUMokt+fcoMAO4nlDs5RjwnKTcqJ1TgBcJc9/MBmYCv6DxHfwthHmhZhGmyb4L+Ey0/wXAzwmTho0DrgIe64C4XIrzuYBc2oomkRtGKKixDpgGHAXeAD4B7ASONFdxStLthIm7jsat2mRms6JtXiIUbRlr0S+bpG8CXzCzYZLGEGa5vNrM/hCtz4/O/VUzWyrpCaDIzJqc9jg6R8/Y9ZJWAhVm9leSLiNUkhphZhVt+Cty3ZyPAbi0ZWZ1wA5JnwbWmtkmSbOByoaLcSscI0xJHetk3M9rrPEnrdXAP0vqS/hEXh8ta2hXjaTNwPho0aXAuSqfxU+JvBcoiN6/SSimUiJpRfT+12ZWdY5jum7OE4BLW5LeAgoJUwtnSDpC+J3Iit5XmNmElo5BGAPY1klNbMvt+QdN7JsBYGanJc0ndB3NB/4S+J6kq80sLWpIu6b5GIBLZx8nfHp/D/hs9L6E0H8eW2jmfM1QbMGGcCHea2a1wNvEVbWK7gwmAVuiRRuBuefTAAtWm9ndwHTCHcJnzueYLvX5HYBLW2ZWEQ2QDgaeJHxqngD8xsz2tfIwio4Rr8rMTkfvhwA/iAaUJxEqeP1L1Iatkp4EHpC0hFDk+zuEOq8/i/a/F1gj6UHgv4ATwBxgRWseVZU0E5gHLAcqCV1KwzmbYFya8gTg0t01hP7/E5LmALvbcPGHUJy8qe3HEJ7aAXgCyAReIySZh4H/iNn2DuAHwFOECl+rgIVmdhzAzN6QNI/wdNIawhjDOuDZVraxhvD00N8QiojvAv7ZzB5v5f6um/KngJzrRNETOiVm9qVkt8W5eD4G4JxzacoTgHPOpSnvAnLOuTTldwDOOZemPAE451ya8gTgnHNpyhOAc86lKU8AzjmXpv4fKdAfqq8k9EQAAAAASUVORK5CYII=","text/plain":["<Figure size 360x360 with 1 Axes>"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["# Plot training and validation loss\n","import matplotlib.pyplot as plt\n","\n","fig, ax = plt.subplots(figsize=(5,5))\n","\n","lw = 2\n","\n","plt.tight_layout()\n","ax.plot(np.linspace(1, len(training_loss), num=len(training_loss)),training_loss, color='rebeccapurple',\n","         lw=2, linestyle='-', label='Training Loss')\n","\n","ax.plot(np.linspace(1, len(training_loss), num=len(training_loss)),validation_loss, color='r',\n","         lw=2, linestyle='-', label='Validation_loss')\n","\n","ax.set_xlabel('# Epochs',fontsize=14)\n","ax.set_ylabel('# Loss',fontsize=14)\n","ax.legend(loc=\"best\",fontsize=12)\n","\n","ax.tick_params(axis='x', labelsize=13)\n","ax.tick_params(axis='y', labelsize=13)\n","\n","\n","plt.grid(color='gray', linestyle='-', linewidth=0.1)\n","\n","\n","plt.show()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"executionInfo":{"elapsed":24,"status":"ok","timestamp":1636212463052,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"hDbJz7X5xIVE","outputId":"f3f93b10-2890-4cb6-87f6-6444f86b35b4"},"outputs":[{"name":"stdout","output_type":"stream","text":["RNNClassifier(\n","  (rnn): RNNCell(59, 128)\n","  (fc): Linear(in_features=128, out_features=1, bias=True)\n","  (activation): Sigmoid()\n",")\n"]}],"source":["# Create RNNClassifier model object\n","\n","model=RNNClassifier(n_features,recurrent_units,device)\n","print(model)\n"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":25245,"status":"ok","timestamp":1636212488277,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"rfH4BX_TxIVE","outputId":"61f99dbe-2714-4dee-f0fa-f13185452959"},"outputs":[{"name":"stdout","output_type":"stream","text":[" Epoch: 0.0 Training Loss 0.478576 Validation Loss 0.4322 Validation AUC 0.71223\n"," Epoch: 1.0 Training Loss 0.405695 Validation Loss 0.3899 Validation AUC 0.78999\n"," Epoch: 2.0 Training Loss 0.364093 Validation Loss 0.3696 Validation AUC 0.80546\n"," Epoch: 3.0 Training Loss 0.353627 Validation Loss 0.3679 Validation AUC 0.80891\n"," Epoch: 4.0 Training Loss 0.351000 Validation Loss 0.3664 Validation AUC 0.81106\n"," Epoch: 5.0 Training Loss 0.349177 Validation Loss 0.3652 Validation AUC 0.81252\n"," Epoch: 6.0 Training Loss 0.347726 Validation Loss 0.3643 Validation AUC 0.81376\n"," Epoch: 7.0 Training Loss 0.346508 Validation Loss 0.3636 Validation AUC 0.81476\n"," Epoch: 8.0 Training Loss 0.345450 Validation Loss 0.3629 Validation AUC 0.81572\n"," Epoch: 9.0 Training Loss 0.344504 Validation Loss 0.3623 Validation AUC 0.81680\n"]}],"source":["# Train RNNClassifier\n","\n","model=model.to(device)\n","model, training_loss, validation_loss=trainer(model,(T,L),(Val_T,Val_L),device,lr,stored_name='rnn',epochs=10)"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":386},"executionInfo":{"elapsed":19,"status":"ok","timestamp":1636212488279,"user":{"displayName":"Anshul Thakur","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"11719449395424920327"},"user_tz":0},"id":"Mu9R8dUAxIVF","outputId":"88a70b17-dff8-4eed-abd5-eaf2b9503f95"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXcAAAFxCAYAAACImejjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUVd748c930sukh5YGiBQVa1ARdbGBugiKbRVc7GXX3+rqqrjqPuq6rq7iPs/jPopYVlRwRXFVsKDYsdBcRQWpkoQESO89Ob8/7kwIYZLMJFOSzPf9et3XzJx77p1zKN975txz7hFjDEoppQYWW6ALoJRSyvs0uCul1ACkwV0ppQYgDe5KKTUAaXBXSqkBKDTQBQiElJQUM3z48EAXw23OEU0iEuCS+JfWO7jqDcFb997Ue/369cXGmNSO6UEZ3IcPH866desCXQy3NTQ0ABARERHgkviX1ju46g3BW/fe1FtEclyla7eMUkoNQBrclVJqANLgrpRSA5AGd6WUGoCC8oaqUv1BZWUlhYWFNDU1BboofqOjZQ6sd1hYGIMGDSIuLs6jc2pwV6oPqqqqoqysjLS0NKKiooIm2LW2tgJgswVXp0Jn9TbGUFdXR35+PoBHAT64/gSV6idKS0tJS0sjOjo6aAK7OpCIEB0dTVpaGoWFhR4d69fgLiIhIvKIiBSJSJWILBWRFDeOu0FEjIjc3SH9bBFZLyIVIlIgIo+LSKTvaqCUfzQ3NxMVFRXoYqg+IioqyuPuOX+33OcCM4DjgHRH2otdHSAiWcCtwPcd0gcBrwPPAonAscBk4B6vllipANEWu3Lqyb8Ff/e5Xwvcb4zZASAitwPbRCTLGONylhVW8L4LuKFDejoQATxrjGkFdonIcuCI7gphjGmbEdYf9KeyelMw19sY09YPG0ycNxaDre7u1NvTuOW3lruIJACZwHpnmjFmO1BJJwFZRK4Daowxr7jY/S3wLnCdiIQ6WvjTgTc6Ode1IrJORNYVFRX1rjJKKa84++yzWbhwodfzKv+23O2O14oO6eXAAbeARSQTuBs43tXJjDGtIvI88DjwGBACLAL+2Un+BcACgOzsbOPpMxwqS+vYm1POwUcN9eg4bwq25204BWO9RaTPjhiJjY1te19bW0tERAQhISEAPPXUU8yaNcvtc7333nv7fe5qtEzHvN7yySefMHv2bHbt2uWT87vDnVFCIuLR/wV//uupcrzGd0hPwGq9d/QM8IAxJt/VyUTkFGAhcAVW98wQrIvE894obHuN9c3cfuZLPHb9cpoamr19eqX6lerq6rYtMzOTZcuWtX1uH9ibm/X/SiD5LbgbY8qBXOBoZ5qIjMQKyBtcHHIG8KCIFItIMTAJuFNEPnfsPwbYYIx5xxjTYozZCzwNnOPtsodHhjJkeAKtLYZdW0u9fXqlBoRPPvmE9PR0Hn74YYYMGcIVV1xBWVkZ06ZNIzU1lcTERKZNm7ZfC3ny5Mk888wzADz//POcfPLJ3HbbbSQmJjJixAjefffdTvOeeOKJ/OEPf3CZ9+eff+bkk0/Gbrdz+umn89vf/pbZs2d7XKdNmzYxefJkEhISOPTQQ3nrrbfa9r3zzjsccsgh2O120tLSePTRRwEoLi5m2rRpJCQkkJSUxEknnRSQewj+vqG6ALhDRD4GSoCHgRXGmJ0u8mZ0+Pwq8Dkwz/H5K+B+EZkCfAAkA9fQrk/fm7LGprB7Rxm5PxUz4rBBvvgKpTp1XfYCv3zPU+uu7dXxe/bsobS0lJycHFpbW6mtreWKK65gyZIltLS0cOWVV3LjjTfyxhsub42xevVqLrvsMoqLi1mwYAFXXXUV+fn5LkeLrF69mjlz5rjMe+mllzJp0iRWrlzJmjVrOPvss5k+fbpHdWlqauKcc87hyiuv5P3332fVqlXMmDGDdevWMWbMGK666iqWLFnCSSedRFlZGT///DMA8+bNIz09Hee9va+//jogI5/83an3ELAMWAvkY/WTzwYQkVkiUu3MaIzZ1X4DGoBKRwsdY8wXWCNo5mH142905LncFwXPHGcNx8/9qdgXp1dqQLDZbNx3331EREQQFRVFcnIy559/PtHR0djtdu666y4+/fTTTo/PysrimmuuISQkhDlz5rB792727t3rUd7c3FzWrl3L/fffT3h4OCeeeKLHgR2soFxdXc3cuXMJDw/n1FNPZdq0abz88suA9ViAjRs3UllZSWJiIkcffXRb+u7du8nJySEsLIyTTjopIMHdry13Y0wL8AfH1nHfIqwbop0dO9lF2kKsfnefyxxrBfecTTrSRvlfb1vU/pKamkpk5L55hLW1tfz+97/nvffeo6ysDLAerdDS0tJ2E7a9IUOGtL2Pjo4GrD5+VzrLW1xcTFJSUlsaQEZGBnl5eR7VpaCggIyMjP1ucmZlZbU9CmDp0qU88MADzJ07l8MPP5yHHnqIiRMnctttt3HvvfcyZcoUAK699lrmzp3r0Xd7Q9+8Hd8HZYxJQQQKtpfR1NgS6OIo1Sd1bKHOmzePzZs3s3r1aiorK/nss8+AfeO6fWHo0KGUlpZSW1vbluZpYAcYNmwYeXl5+/WX5+bmkpaWBsCECRN48803KSws5Nxzz+Wiiy4CwG63M2/ePHbs2MFbb73FY489xocfftjLWnlOg7ubIqPDGJyVQEtzKwXb9aaqUu6oqqoiKiqKhIQESktLue+++3z+nVlZWWRnZ3PvvffS2NjIV199xbJly7o9rr6+fr/t2GOPJTo6mr/97W80NTXxySefsGzZMn71q1/R2NjIokWLqKioICwsjLi4uLYW/vLly9m2bRvGGOLj4wkJCQnIsFYN7h7Y1zWj/e5KuePmm2+mrq6OlJQUjj/+eM4880y/fO+iRYv46quvSE5O5u677+biiy/ucox4fn4+UVFR+215eXksW7aMd999l5SUFH7zm9/wwgsvMHbsWABefPFFhg8fTlxcHPPnz2fRIqtXeevWrZx++unExsYyceJEfvOb33DKKaf4pd7tiS9/HvVV2dnZpicLZH/w0gZe+++vOem8scy+62QflMw1XTQ4+Oq9fft2DjnkkEAXxe989cjfiy++mLFjx/rll0NPuFPvTZs2MW7cuAPSRWS9MSa7Y7q23D2Q5Rwxs7kkwCVRSnVl7dq1bN++ndbWVt577z3efPNNzj333EAXy690sQ4PZIyxgnv+1hKam1oIDTvwbr9SKvD27NnDzJkzKSkpIT09nSeffJKjjjoq0MXyKw3uHoiKDWdQZjyFuRUUbC9r64NXSvUt55xzDuec4/XJ6v2Kdst4KGusTmZSSvV9Gtw9pDNVlVL9gQZ3D+lMVaVUf6DB3UPO4L5rayktzcG1WoxSqv/Q4O6haHsEqelxNDe2sPvnskAXRymlXNLg3gM6U1Up1ddpcO8BZ3DP1X53pTwiImzbtg2A66+/nj//+c9u5fXUokWL2p7K6As7d+5ERPr0alMa3HtAZ6qqYHbmmWfypz/96YD0N998kyFDhrgd8ObPn88999zT6/K4CrSzZs3i/fff7/W5+zMN7j3gbLnnbS7Wm6oq6MyZM4eXXnrpgMf2vvjii8yaNYvQUJ0b2RdocO+BmPhIkofZaWpoYc/O8kAXRwUDEf9sbjj33HMpKSnh888/b0srKytj+fLlTJ8+nYkTJ5KQkMDQoUO58cYbaWxsdHmeyy+/nLvvvrvt8yOPPEJaWhrp6ek899xz++V9++23Oeqoo4iLiyMjI4N77723bd/JJ1sP8UtISCA2NpavvvqqbY1Vpy+//JIJEyYQHx/PhAkT+PLLL9v2TZ48mXvuuYdJkyZht9uZMmUKxcWe3U8rKChg+vTpJCUlMWrUKJ5++um2fWvWrCE7O5u4uDgGDx7MLbfcAliPGJ49ezbJyckkJSVx3HHHdbrqVE9ocO8hnamqglVUVBQXXXQRL7zwQlvakiVLGDt2LLGxsfz973+nuLiYr776ig8//JAnnnii23O+9957PProo6xYsYLNmzezcuXK/fbHxMTwwgsvUF5ezttvv82TTz7Ztg6rcwGQ8vJyqqurmThx4n7HlpaW8stf/pLf/e53lJSUcMstt/DLX/6SkpJ93aqLFy/mn//8J4WFhTQ2NrYtdu2uX/3qV6Snp1NQUMBrr73GH//4Rz766CMAbrrpJm666SYqKyvZvn1726IeCxcupKKigry8PIqKinjiiSeIiory6Hu7osG9h3SmqvIrY/yzuWnOnDm89tpr1NfXA/DCCy8wZ84cjjnmGI4//nhCQ0MZPnw41113XZdrpjotWbKEK664gsMOO4yYmJj9WuZgta7Hjx+PzWbj8MMP55JLLnHrvGC1+g8++GAuu+wyQkNDueSSSxg7dux+C3hcccUVjB49uu3C9e2337r9Z5GXl8cXX3zBww8/TGRkJEceeSRXX31128UvLCyMbdu2UVxcTGxsLMcff3xbeklJCdu2bSMkJIRjjjmGuLg4t7+3Oxrce0hnqqpgduKJJ5KSksIbb7zB9u3bWbNmDZdeeilbtmxh2rRpDBkyhLi4OP74xz+61cXhXK/UKSsra7/9q1ev5pRTTiE1NZX4+Hjmz5/vdtdJQUHBAedrvxYqHLgea2frtnZ2/qSkJOx2u8vzP/vss2zZsoWxY8cyYcIEli9fDsBll13G1KlT21r9d9xxB01NTW5/b3c0uPdQ1rhUAPI2l9DaojdVVfD59a9/zQsvvMBLL73E1KlTGTx4MDfccANjx45l69atVFZW8uCDD7q1XurQoUP3W+c0Nzd3v/2XXnop06dPJy8vj4qKCq6//vq283Zct7WjYcOGkZOTs19a+7VQe2vYsGGUlpZSVVXl8vwHH3wwL7/8MoWFhdxxxx1ccMEF1NTUEBYWxn/913+xceNGVq1axdtvv71fV1dvaXDvodiESJKGxNJY38ze3IpAF0cpv/v1r3/NypUrefrpp5kzZw5grZkaFxdHbGwsP/30E08++aRb57rooot4/vnn2bhxI7W1tQesmFRVVUVSUhKRkZGsWbOGxYsXt+1LTU3FZrOxY8cOl+c+++yz2bJlC4sXL6a5uZlXXnmFjRs3Mm3atB7WfH8ZGRmccMIJ3HnnndTX17NhwwaeffZZZs+eDcBLL71EUVERNpuNhIQEwFpx6eOPP+b777+npaWFuLg4wsLCvLoClQb3XnD2u+tMVRWMhg8fzgknnEBNTQ3Tp08H4NFHH2Xx4sXY7XauueYaLr74YrfOddZZZ3HzzTdz+umnM3r0aE499dT99j/xxBP86U9/wm63c//997fdlASrG+Wuu+5i0qRJJCQk8PXXX+93bHJyMsuXL2fevHkkJyfzt7/9jeXLl5OS4r31GF5++WV27tzJsGHDOO+887jvvvs4/fTTAetm8aGHHkpsbCw33XQT//rXv4iKimLPnj1ccMEFxMXFceihh3LyySdz2WWXea1MuoZqL7z9zDe8NX8dp11yGBfdeoIXSuZaMK8lCsFZb11DNbjanbqGah+jM1WVUn2VBvdeaD9TtbU1+H4BKTWQLVq0iNjY2AO2Qw89NNBFc4sG916IS44mYVAM9TVNFOpNVaUGlFmzZlFdXX3A9uOPPwa6aG7R4N5LOlNV+YqzH1apnvxb0ODeSzpTVflCVFQU+fn5NDY2ujVOXA1MxhgaGxvJz88nJibGo2P18W29pDNVlS8MGTKEqqoqcnJy+vQzw73N3YlJA01X9Q4NDSU+Pt7joZsa3HvJOVM19yfrpqrNFlz/KJVv2Gw2Bg0axKBBgwJdFL8K5uGv4N16a7dML8WnRBOfEk19TRPF+ZWBLo5SSgEa3L1CZ6oqpfoavwZ3EQkRkUdEpEhEqkRkqYh025EkIjeIiBGRuzukh4rIfSKSIyI1IrJdRM7yXQ1c0zVVlVJ9jb9b7nOBGcBxQLoj7cWuDhCRLOBW4HsXu+cDU4CpQCxwErDJW4V1V9twSJ2pqpTqI/x9Q/Va4H5jzA4AEbkd2CYiWcaYnE6OeRa4C7ihfaKIjAGuAsYZY35yJBe4UwhjTNsNDG8YcpD1gP3cTUXU19d7/U6/N8van2i9g0+w1t0X9fZby11EEoBMYL0zzRizHagEjujkmOuAGmPMKy52n+I49iIRyReRXBF5UkTsLvIiIteKyDoRWVdU5N3uk/iUaOyJkdRWNVJS4P5D/pVSylf82XJ3Bt2O8/TLgQPWlhKRTOBu4PhOzpfiOG6cY4sBXgceA67pmNkYswBYANZTIb091CprXCo/fJnHnh2VpI303qNE2wu24WFOWu/gE6x1769DIZ3LlMR3SE/AaoF39AzwgDEm38W+9ue7xxhTaYzZDTyM1afvdzpTVSnVl/gtuBtjyoFc4GhnmoiMxGp9b3BxyBnAgyJSLCLFwCTgThH53LHfuYJtx7nZAZmr7ZzMpDNVlVJ9gb9vqC4A7hCRj4ESrJb2CmPMThd5Mzp8fhX4HJjn+Pw51gia+xx98zHAbVhdM36X2e4BYsaYoJs+rZTqW/w9FPIhYBmwFsgHQoDZACIyS0Ta7kYaY3a134AGoNIYs9exvxU4B0gC9gL/wbpZ+wc/1qdN4uAYYhMiqalooHSP3lRVSgWWX1vuxpgWrOB7QAA2xiwCFnVx7GQXaTnA2V4sYo+JCJnjUtj41S5yNhWTPNTloB2llPILffyAF+lMVaVUX6HB3YucM1VzdMSMUirANLh7UaZjxEye46aqUkoFigZ3L0oeGkt0XARVZfWU7a0JdHGUUkFMg7sXiYiuqaqU6hM0uHuZzlRVSvUFGty9TGeqKqX6Ag3uXtZxpqpSSgWCBncvS0mzE20Pp7Kkjori2kAXRykVpDS4e5mIkDFW11RVSgWWBncf0JmqSqlA0+DuAzpTVSkVaBrcfaD9TFWllAoEDe4+kJoeR2RMGOVFtXpTVSkVEBrcfcBmEzLH6GQmpVTgaHD3EZ2pqpQKJA3uPqIzVZVSgaTB3Ucy9QFiSqkA0uDuI4My44mIDqNsbw2VpXWBLo5SKshocPcRm03IGJMMaOtdKeV/Gtx9SGeqKqUCRYO7D+lMVaVUoGhw9yGdqaqUChQN7j40JCue8MhQSnZXU11eH+jiKKWCiAZ3H7KF2MgYrTdVlVL+p8Hdx5wzVXUyk1LKnzS4+5hzpqq23JVS/qTB3cd0pqpSKhA0uPvYkOEJhEWEUJxfRU2F3lRVSvmHBncfCwm1ke68qbq5JMClUUoFCw3ufqAzVZVS/qbB3Q90pqpSyt80uPuBzlRVSvmbX4O7iISIyCMiUiQiVSKyVERS3DjuBhExInJ3J/sPF5FGEVnp/VL33rCRiYSGh1CYV0lddWOgi6OUCgL+brnPBWYAxwHpjrQXuzpARLKAW4HvO9kfCjwHfO69YnpXSKiN9FFJgA6JVEr5R6ifv+9a4H5jzA4AEbkd2CYiWcaYnE6OeRa4C7ihk/13AmuBvcCJ7hTCGENDQ4NHBe+ttNGJ7NxYxPbvdzN8fLJHx/q7rH2F1jv4BGvdfVFvv7XcRSQByATWO9OMMduBSuCITo65DqgxxrzSyf7xwOXAHW58/7Uisk5E1hUV+X/UinPhjl1bSv3+3Uqp4OPPlrvd8VrRIb0ciOuYWUQygbuB412dzNEd80/gJmNMpYh0+eXGmAXAAoDs7GwTERHhUeF766DxQwHYtbmUnn63v8vcV2i9g0+w1t2b9fZnn3uV4zW+Q3oCVuu9o2eAB4wx+Z2c73ZgqzFmuZfK51PDDkokNMzG3twKvamqlPI5vwV3Y0w5kAsc7UwTkZFYrfYNLg45A3hQRIpFpBiYBNwpIs4bp1OAX7bbfztwsuNzki/r0hOhYSEMc9xUzduiM1WVUr7l79EyC4A7RGSEiMQBDwMrjDE7XeTNwOqLP9KxrQP+D7jAsf9C4JB2++cDqx3vy31Yhx7L0pmqSik/8fdomYeARKzRLRHAB8BsABGZBTxljIkFMMbsan+giDQAlcaYvY79RR32VwINHY/rSzJ1pqpSyk/8GtyNMS3AHxxbx32LgEVdHDu5m3Pf28vi+ZxzpmruJg3uSinf0scP+FHaqCRsIcLenHLqa5sCXRyl1ACmwd2PwsJDSDsoCWNgl95UVUr5kAZ3P9M1VZVS/qDB3c90TVWllD9ocPezfQt3aHBXSvmOBnc/c95U3b2znIY6vamqlPINDe5+Fh4ZyrCRiZhWozdVlVI+o8E9ANq6ZrTfXSnlIxrc3WEMbNgACxd65XQ6U1Up5Wv+fvxA/1RSAkcdBTYbnHMOJPXuuWQ6U1Up5WvacndHSgqceio0N8O//93r02WMTkZswu6fy2isb/ZCAZVSan8a3N118cXW67/+1etThUeGMnR4Aq0thvxtujKTUsr7NLi7a+ZMCA2Fjz6CwsJen05nqiqlfEmDu7uSkmDKFGhthaVLe306namqlPIlDe6e8GLXjM5UVUr5kgZ3T8yYARER8PnnUFDQq1Olj05GBPK3l9LUoDdVlVLepcHdE/HxcNZZ1rj3V1/t1akio8MYojdVlVI+osHdU7/6lfXqza4Z7XdXSnmZBndPTZsG0dHw9deQk9OrU+lMVaWUr2hw91RMjBXgAZYs6dWpdKaqUspXNLj3hJe6ZjLHJAOQv62U5qaW3pZKKaXaaHDvibPOArsdvvkGtm3r8WkiY8IZnBlPS3MrBdvLvFhApVSw0+DeE5GR1rBIgFde6dWpdKaqUsoXNLj3lJe6ZrK0310p5QMa3HvqjDMgMRF++AE2buzxaXQ4pFLKFzS491R4OJx3nvW+F10zzuC+a1spLc2t3iiZUkppcO+V9l0zxvToFFGx4QzKiKO5sYWC7TpTVSnlHRrce+OUUyA1FbZsge++6/FptGtGKeVtGtx7IzQUzj/feu+FrhmdqaqU8hYN7r3lha4ZnamqlPI2De69deKJMHQo7NwJa9f26BRtN1W3luhNVaWUV2hw762QELjwQut9D7tmYuIiSEmz09TQwp6d5V4snFIqWPUouItIlIicLiJZ3i5Qv+TsmnnlFWsZvh5o63fXmapKKS9wK7iLyPMi8hvH+3BgDfA+sFlEznL3y0QkREQeEZEiEakSkaUikuLGcTeIiBGRu9uljRaR10Qk33GuH0XkanfL4lXHHw+ZmZCfD19+2aNTOGeq5mi/u1LKC9xtuU8Fvna8nw7YgSHAvY7NXXOBGcBxQLoj7cWuDnD8OrgV+L7DrkTgY2ACEAdcBzwqIjM9KI93iMBFF1nve9g142y55+mIGaWUF4S6mS8RKHS8PxNYaowpFJF/AXd58H3XAvcbY3YAiMjtwDYRyTLGdLbyxbOO77ihfaIxZjWwul3SKhF5D5gMvN5VIYwxNDQ0eFDs7snMmYQ/+ijm1VdpfPhhqy/eA0NG2gHI21JCXW0dtpB9111vl7W/0HoHn2Ctuy/q7W7LfQ9wmIiEYLXiVzrSY4Emd04gIglAJrDemWaM2Q5UAkd0csx1QI0xptvmsIhEA8cDLmcTici1IrJORNYVFXm/X9scdRRm5Ehk717ks888Pj4mPpKkITE01jezN6fC6+VTSgUXd1vuzwGvAAVAC/ChI/044Cc3z2F3vHaMXOVY3Sr7EZFM4G6sgN0lx0XnRSAPeMFVHmPMAmABQHZ2tomIiHCz2B645BL4y18If/11OPNMjw/PGpdK6Z4a9uyoZPi4IQfs90mZ+wGtd/AJ1rp7s95utdyNMfcDV2IFxxONMY2OXc3Aw25+V5XjNb5DegJW672jZ4AHjDH5XZ1URMKAl4GhwDRjjFu/JHzi4out16VLocnzYuhMVaWUt7jbcscYs9RF2kIPji8XkVzgaOBbABEZidVq3+DikDOAY0TkL47P8cAEEZlqjDnJcXwk8BpW99AUY0y1u+XxicMOg3HjYNMm+PBDj1vvOlNVKeUt7g6FvEhEprT7/CcR2SUiK0RkqAfftwC4Q0RGiEgcVqt/hTFmp4u8GVh98Uc6tnXA/wEXOMoQC7wLhANnBTywgzVqpheLeGQ5VmXK21xMa4vOVFVK9Zy7N1Tvdb4RkaOBPwL/C4QB8zz4voeAZcBaIB8IAWY7zjtLRNoCtDFmV/sNaAAqjTF7HVnOxxoZcyJQJCLVjm2+B+XxPmfXzBtvgId3wO2JUSQOjqGhrpnCPFc9VUop5R53g3sWsNnx/jzgDWPM34BbgNPc/TJjTIsx5g/GmBRjjN0YM9MYU+zYt8gYE9vFsZONMQ+0+7zQGCPGmGhjTGy77Xp3y+MTY8bAEUdARQWsWOHx4TpTVSnlDe4G93r2jXY5jX1DISvapSunXnXN6ExVpVTvuRvcPwfmicg9QDbwjiN9NNbwQ9Wec7bqW29Bba1Hh+pMVaWUN7gb3G8EGrFuZl5vjClwpJ8FeN73MNCNHAkTJkBNDbzzTvf522lblWlzMa2tPXs+vFJKuTvOfZcx5hxjzBHGmOfapd9sjPmd74rXj/WwayY+JZqE1Gjqa5ooytOZqkqpnvHokb8icqqI3CgivxWRU3xVqAHB+Yz3t9+Gqqqu83aw76aqds0opXrG3XHuaSKyBvgAuAPr6Y4rRWS1iAzzZQH7rYwMmDQJ6uth2TKPDm3fNaOUUj3hbsv9f7GeKTPKGJNhjMkADnak/a+vCtfv9bBrRmeqKqV6y93gfgbwW2PMz84Ex2N7f+fYp1y54AKw2eC996Dc/eXznDNVc38qxvRw0W2lVHDzpM/dVZTRyNOVIUPgF7+wHiL2xhtuH5aQGkNcchR11Y0U53vWX6+UUuB+cP8QeFxEMpwJjkfy/jf7Hv+rXOlp14zOVFVK9YK7wf13QAywQ0RyRCQH2O5Iu8lXhRsQZs60VmVauRKK3e9D15mqSqnecHecex7Wo3rPBh51bGcZY4527FOdSUmBM86AlhZ4vcvV//bTNmJGZ6oqpXrA7T53Y/nAGPO4Y1spIoeKSGH3Rwc555MiPeiaaR/c9aaqUspTHk1iciEUSPZGQQa0c8+F8HD49FPYs8etQxIHx2BPjKS2soHS3YF/VL1Sqn/pbXBX7khIsFZlam2F115z6xAR2TfefXOJL0unlBqANLj7S0+6ZoDG7IsAACAASURBVMZYP4p2bS71RYmUUgOYBnd/OecciIyEL76APPfuQTtb7nnacldKeajL4C4iVSJS2dkGfOmncvZ/djtMm2a9f/VVtw7Zt6Zqid5UVUp5JLSb/Tf6pRTB4uKLrT73f/0Lbrml2+xJQ2KJiY+gpqKBsr01DM2K9EMhlVIDQZfB3Riz0F8FCQpnnw0xMbB2LezYYS3q0QURIXNsCptW55O3uYShWTowSSnlHu1z96foaJgxw3q/ZIlbh2Rpv7tSqgc0uPubh6Nm2tZU1REzSikPaHD3t6lTIT4evvsONm/uNvu+BbP1pqpSyn0a3P0tIgLOO896/8or3WZPSbMTbQ+nurxeH/+rlHKbBvdAaN81001rXEQYe5y1kuH6lTt8XTKl1ADhdnAXkX+ISJIvCxM0TjsNkpNh0yb44Ydusx9z+ggA1q7Y5uuSKaUGiO4mMaW3+3gpEOtI/779wh3KQ2FhcP751ns3umbGHpdGlD2cXVtLKdhR5uPCKaUGgu5a7j85FudYDEQCzoA+HAjzZcEGPA+6ZsLCQzjiF5mAtt6VUu7pLrgnABcC6x153xGRLUAEMFVEBvu4fAPXL34BgwfD9u3wzTfdZnd2zax7f7uOmlFKdau74B5mjFljjJkH1AFHAVcALcCVwM8i0v14PnWgkBC48ELrvRtdMwcfPYS45CgK8yp16T2lVLe6C+7lIrJaRB4DwoEoY8wXQDNwMZAIXOXjMg5czq6ZV17ptmvGFmLjmNOtxxVo14xSqjvdBfc04AGgAes5NOtF5HOsQH801up7q3xbxAHshBMgPR1yc+Hrr7vNPmHqKADWfbCD1lbtmlFKda7L4G6MKTbGLDPG3AnUAhOAxwGDtUh2hYh86vtiDlA2G1x0kfXeja6ZkeMHkTw0lvLCGrb9Z7ePC6eU6s88ncRUYYxZAjQBpwIjgCfcPVhEQkTkEREpcjwrfqmIpLhx3A0iYkTk7g7po0RkpYjUiMguEbnVw/oEnrNrZskSaGnpMquIkD3lIADWvr/d1yVTSvVjngT3w4Fdjvc5QJMxZo8xpvsm5z5zgRnAcYBzDP2LXR0gIlnArcD3HdJDgGXAJiAVmA7cISIXe1CewJswAUaMgN27YVX3PVzOrpn1K3fQ0tzq69Ippfqp7hbraGOMyWv3/rAeft+1wP3GmB0AInI7sE1EsowxOZ0c8yxwF3BDh/STgSzgTmNMLfCNiDwFXA90ecExxtDQ0NDDKnhfyPnnE/roo7QsXkzz8ccfsL99WVMzYxg8PJ69OyvYsOpnDpmYfkD+gaIv/R35U7DWG4K37r6ot9+eLSMiCUAm1ph5AIwx24FK4IhOjrkOqOnk18ERwBZjTHW7tG+6ONe1IrJORNYVFRX1sBa+0eoYEmn797+hubnLvCLSNuZ9/cqffV42pVT/5HbL3QvsjteKDunlQFzHzCKSCdwNHNiU3Xc+t84FYIxZACwAyM7ONhEREe6V2h8mTIAxY5DNm4n48ks44wyX2ZxlnvjLMbzzzLds+CwPMSGER/rzr9H/+tTflR8Fa70heOvuzXr786mQzufVxndIT8BqvXf0DPCAMSa/i/O5e66+TWT/Me/dGJQRT9YhqTTUNvH9qlwfF04p1R/5LbgbY8qBXKzx8QCIyEislvYGF4ecATwoIsUiUgxMAu50jLMH+A4YLSIx7Y45ypHe/ziD+9Kl0NjYbfYJjlEz63TUjFLKBX8/z30B1oiWESISBzwMrDDG7HSRNwOr//xIx7YO+D/gAsf+z7BG7TwoIlEiciRwHfCUb6vgI4ccAuPHQ3k5fPBBt9mzzxiJCGxYlUtddfcXA6VUcPF3cH8Ia/jiWiAfCAFmA4jILBFpuzlqjNnVfsOaJVtpjNnr2N8CnAMcBpQA7wCPGGPcW5y0L/KgayZxcCyjjhpKc2ML336y07flUkr1O34N7saYFmPMH4wxKcYYuzFmpjGm2LFvkTEmtotjJxtjHuiQts0Yc5oxJtoYM8wY86iv6+BTzuD+xhtQX99t9gk6oUkp1QldZq8vGTUKjjkGqqrg3Xe7zX7M6SOxhQibVu+iqqzODwVUSvUXGtz7Gg+6ZmITIjnkuHRaWwzffKhj3pVS+2hw72ucDxJbtgxqarrNrs+aUUq5osG9r8nKgokTobYWli/vNvuRk4cTFhHCtv/spmxvdbf5lVLBQYN7X9R+fdVuRMWGM35SJsZYz3lXSinQ4N43XXihNWv13XehsvsJtxOmWl0za3SFJqWUgwb3vmjYMDj5ZGhogDff7Db7YZMyiYwJI3dTMXtzOz5uRykVjDS491UedM2ER4Zy5OThgK6vqpSyaHDvq84/H0JC4P33obS02+xtE5pWbMd0s9i2Umrg0+DeVw0aBKeeCs3N2Nzomhl3XDox8RHs2VnOrq3dXwyUUgObBve+zNE1E/Lqq91mDQm1cczpIwHtmlFKaXDv2847D8LCkE8+gcLCbrM711dd9752zSgV7DS492VJSTBlCtLaiu2NN7rNPurIISQOjqFkdzU7Nuz1QwGVUn2VBve+ztk1s2RJt1ltNiH7DOeYd30cgVLBTIN7XzdjBiYiAvniC1i/vtvszmfNrF+5g5bmVl+XTinVR2lw7+vi4mi55hrEGJg1q9uHiWWNS2FQRhxVpXVsWV/gp0IqpfoaDe79QMuf/0zrIYfA5s1wyy1d5hWRthura7VrRqmgpcG9P4iKonnhQoiIgAULrJWauuB81sw3H/1MU2OLP0qolOpjNLj3E2b8eHj4YevD1VdDQeddLkNHJJI+Opm66kZ+/DLPTyVUSvUlGtz7k//3/2DqVCgpgTlzoLXzG6b7HkegE5qUCkYa3PsTmw2efx5SU2HlSvj73zvN6hw1s+HzXBrqmvxUQKVUX6HBvb8ZMgSee856f+ed8O23LrOlDLMz8vDBNNY3892nOX4soFKqL9Dg3h9Nmwa/+Q00NcEll1hL8rmgXTNKBS8N7v3Vo4/CuHHw009w660us2SfMRKxCT9+tYuaino/F1ApFUga3PurqCh4+WUID4f58+Gttw7IEpcczdjsYbQ0t/Kfj3f6v4xKqYDR4N6fHXEEPPSQ9f6qq2D37gOyOMe8r31fJzQpFUw0uPd3N90EU6ZAcTFcfvkBwyOPPGUEIaE2Nq8roKLYdd+8Umrg0eDe3zmHRyYnW0vy/c//7Lc7Ji6Cw07IwLQa1q/cEZgyKqX8ToP7QDB06L7hkXPnwnff7bfb2TWzRkfNKBU0NLgPFNOnw/XXQ2MjXHop1NW17Tr85CzCI0P5+ftCivMrA1hIpZS/aHAfSObNg7FjYeNGuO22tuSIqDCO+EUWoDdWlQoWGtwHkuhoWLwYwsLg//4Pli9v29V+fVWl1MCnwX2gOeoo+OtfrfdXXAF79gBwyPHpRNvD2bW1lIIdZQEsoFLKH/wa3EUkREQeEZEiEakSkaUiktJJ3pNE5BsRKRWRCsf7mR3ynC0i6x37C0TkcRGJ9E9t+rDf/x5OP32/4ZFh4SEcdeoIQB9HoFQw8HfLfS4wAzgOSHekvdhJ3s3AeUAykADcDLwkIuMARGQQ8DrwLJAIHAtMBu7xUdn7D5sNFi6EpCRYsQIefxxgvxWajDGBLKFSysdC/fx91wL3G2N2AIjI7cA2Eckyxuz36EJjTKHzvYjYgFasi9EoYBPWxSECeNYY0wrsEpHlwBHdFcIYQ0NDg5eq5Hs9KmtyMrb58wm76CLM7bfTNGkSww87lLjkKIp2VbL1uwKyxrn80dRn9Ke/I28K1npD8NbdF/X2W8tdRBKATGC9M80Ysx2opIuALCLlQAPwObAaeN+x61vgXeA6EQkVkSxgOuByDToRuVZE1onIuqKiIi/UqO9rnT6dlquuQhobCZ0zB1tjA0edOhyA9R/8HNjCKaV8yp8td7vjtaJDejkQ19lBxpgEEYkAzgLGAM2O9FYReR54HHgMCAEWAf/s5DwLgAUA2dnZJiIioscVCZQelfl//gdWrcK2cSMRf/oTx19zF5++uolvP9rJxbdOwmYT7xfUy/rj35U3BGu9IXjr7s16+7PPvcrxGt8hPQGr9d4pY0yDMeYN4BfA1QAicgqwELgCq3tmCNZF4nnvFXkAiInZNzzy8ccZkbuO5GF2yotq2fafAx80ppQaGPwW3I0x5UAucLQzTURGYgXkDW6eJhQ42PH+GGCDMeYdY0yLMWYv8DRwjvdKPUAcfTT85S8AyJVXcuLxCYB1Y1UpNTD5e7TMAuAOERkhInHAw8AKY8zOjhlF5HwRGe/oT48UkWuAU4EVjixfAeNFZIpYUoBraNenr9q59VY49VQoLOSU9x8DY1j/4Q5amjtfZFsp1X/5O7g/BCwD1gL5WP3kswFEZJaIVLfLOxRrqGM5UABcCVxijPkAwBjzBXADMA+rH38j1o3Xy/1RkX7HZoMXXoDERKI+XcmM1tXUVDSw8etdgS6ZUsoH/BrcHd0nfzDGpBhj7MaYmcaYYse+RcaY2HZ5/2GMOdgYE2uMSTLGTDTGvNrhfAuNMeONMXHGmEHGmAuNMXn+rFO/kpYGzzwDwNQfFjGsLl8nNCk1QOnjB4LNzJlw9dWENDdy9Y5n+OHjbTTWNwe6VEopL9PgHoz+/nc4+GDS6gv45dZX+H5VbqBLpJTyMg3uwSg2FhYvpjUklNMKP2LP/JcDXSKllJdpcA9W2dnUz7Uew3PSW3+j7me9sarUQKLBPYhF33cXuUPHE9dUSd3FvwZ9mJhSA4YG92AWEsLu//o7NSHRJK39GJ58MtAlUkp5iQb3IHfoxZNYNOIyAMytt8KPPwa4REopb9DgHuRiEyJpOHsGq5InIfX11uLa9fWBLpZSqpc0uCsmTDmIJRkXURY3FDZsgD/+MdBFUkr1kgZ3xZGTh9MaHcOTwy7HhIZa4+Dff7/7A5VSfZYGd0VkTDjjT8wkJ2Y428693kqcMweCZFETpQYiDe4K2Le+6qthJ8PJJ8OePXDVVTo8Uql+SoO7AmD8pAwiY8LI2VJG0V//AQkJsGwZPPVUoIumlOoBDe4KgLCIUI46ZQQAq39s3BfUb7kFNm0KYMmUUj2hwV21yZ5yEGCt0GQuvBAuvxzq6mDGDLjzTpg/H957zwr2tbWBLaxSqkv+XCBb9XHjjk0jNiGSPTvL2bWlhIz//V9YtQq2boWHHjrwgJQUyMqytuHD9713bgkJIH1/AW6lBiIN7qpNSKiNY04fyaevbWTNiu1k/O44WLMG3noLcnKsbedO6zU3F4qLrW19Jysb2u2ug75zGzxYg79SPqLBXe1nwtSD+PS1jax7fzsz/9+xSGKiNSyyo9ZWa0SNM+h3DP45OVBVBd9/b22uREZCZuaBQd95QUhOhlD9J6pUT+j/HLWfg44YQuLgGEr3VLNjw14OOmKI64w2GwwbZm0TJx643xgoLT0w+Le/AJSWwpYt1uZCeEiIdf70dGuJwLQ063PH9zEx3vsDUGqA0OCu9mOzCdlTDuKDFzewZsX2zoN7d0SslndyMhx9tOs81dWug7/jAiC7d0NenrV1JT7eddBv/37wYP0VoIKK/mtXB5jgCO7rV+7golsmEhLqo0FVsbFw6KHW5kJDZSXk5xNRXAz5+VBQYL12fF9RYW1dDdm02awA39UFIC3NulDofQA1AGhwVwfIHJvCoMx4CnMr2LyugEOOTw9MQSIiYORIGDeu8zzGQFnZvkDf2UWgsBB277a2des6P19U1L5AP3QopKZao4Kcr84tNdX6VRIW5v16K+UFGtzVAUSECVMP4u2nv2Htim2BC+7uEIGkJGsbP77zfE1N1g3g7i4C1dWwbZu1uSM+/sDg7+pC4HwfH2/9ilDKxzS4K5eOnTqKt5/+hv98vJNL72whLDwk0EXqnbAwyMiwtq5UVu4L9Lt37xvuWVS07337zdkltH27e+UICdk/8Lu4GEh8vPWrYPBga65AXJzeL1Ae038xyqUhwxPIGJ1M3pYSfvwyjyMnDw90kfwjLs7axo7tPm9rK5SXd34BcPW5shL27rW2ToS7SrTbrUDvzhYff+BnvTgEHf0bV52aMPUg8raUsHbFtuAJ7p6w2fZ1CY0e7d4xjY1QUtLlBaC1sBBKSrBVVloXj/Jya85AVVX3I4c6Exvr/sUhIcG6mNjt1oXObreO1+6kfkWDu+pU9pSDeP3xNXz3WQ5FuypJTY8LdJH6v/Bw60bt0KGdZmlqaAAgIiLCSmhtte4FOAN9T7bqamvbtavnZY+NPTDo9+S93a6/JPxA/4RVp5KH2hl3XBqbVufzwKylzLrzJI49c1SgixV8bLZ93UWZmZ4f35OLQ1WV1YXk/MXgvDhUV1v3InorKsrlBSA0Otq6iMTHW68xMdarc+vsc0yM/rLoQEwQLsaQnZ1t1nU1HK6PaejYkvOjmsoGXnrgM7756GcAJk4bza9uO4HIGJc9w14VyHoHUp+st/MC0THod/a+u3y+iDvR0e5fDFx9jolxvYX4fjBBb/7ORWS9MSb7gHQN7n1foP+zG2NY9cZPvPLolzQ1tDAoI46rHzyNrHGpPv3eQNc7UAZ8vY2xHhntIug3lZZCTQ1hDQ37finU1Oz/y6Gzz74SHn5gwHdeSNxJ7ypvdDSIaHD3Fg3uPVOwo4xn7/qQXVtLCQm1cd6Nx3LapeOx2Xwzo7Ov1NvfgrXe0Iu6t7Zaaw94cjFw9bmm5sDN1zEyOhoTE0PrWWcRsnChx4d3Fty1z125bdjIROY+fy6vP76Gj/71A6/999ds/HoXl987mfiU6EAXTwUzm21fi9ibjIGGhv2DfW2t64uAq/Tu8tbVQW0t4vzsRdpy7wf6Ykvuu89yWHjfJ9RUNGBPiuLyeydz2AndTBDyUF+stz8Ea70hCOve2gq1tTSUloLNRkS657PBO2u5+/X2soiEiMgjIlIkIlUislREUjrJe5KIfCMipSJS4Xg/s0OeUBG5T0RyRKRGRLaLyFn+qU1wO+LkLP70rwsYM2EYVaV1PP67d3n171/R1NgS6KIp1X/YbNbN3MGDrZnK3jy1V8/WvbnADOA4wHmJerGTvJuB84BkIAG4GXhJRNo/RWo+MAWYCsQCJwG6mrOfJKTGcPM/zua8G4/FFiKsXPQ9f7vyTfbmlAe6aEoFPX/3uV8L3G+M2QEgIrcD20QkyxiT0z6jMabQ+V5EbEAr1sVoFLBJRMYAVwHjjDE/ObIWuFMIY0zbz7/+oK+X9ZRLxjHi8BQW3vsZuT8V88Ds17nw98dx7NkHIb14fG5fr7evBGu9IXjr7ot6+63lLiIJQCbQtuCmMWY7UAkc0cVx5UAD8DmwGnjfsesUx7EXiUi+iOSKyJMiYu/kPNeKyDoRWVdUVOSVOql9hh+ayu3/PIdjzhhBY10zix78goX3fkZddWOgi6ZUUPJny90ZdCs6pJcDnc5rN8YkiEgEcBYwBmh27EpxHDfOscUArwOPAde4OM8CYAFYN1T74w2bvl7miIgIrnnwdMZP2srLD6/im5U7ydlYwtV/OZWR4wf36rzBKFjrDcFbd2/W25997lWO1/gO6QlYLfBOGWMajDFvAL8Aru5wvnuMMZXGmN3Aw1h9+ipARISJ00Zz10szyRybQklBFY9c/RbvPPcfWltaA108pYKG34K7MaYcyAXaFtQUkZFYre8Nbp4mFDjY8f5b56k7flUviqm8ZHBWAnf8cwZnzD6c1hbDm0+s5b9/+w5lhd4dy6uUcs3fo2UWAHeIyAgRicNqaa8wxuzsmFFEzheR8Y7hjpEicg1wKrDCkeVz4HvgPhGJEZFBwG1YXTOqDwgNC+GCm4/nd4+fRVxyFJvXFfDnS17ju89yuj9YKdUr/g7uDwHLgLVAPhACzAYQkVki0v4BEUOxAnU51iiYK4FLjDEfABhjWoFzgCRgL/AfrJu1f/BLTZTbDp2YwT2Lz+fQEzKoqWjgiVtW8PLfvqCpobn7g5VSPaIzVPuBgTJrr7XV8OHi7/n3P9bQ0txK2qgkrn7wNIaNTHSZf6DU21PBWm8I3rr74sFh+gBk5Tc2m3DG7MOZ+/y5DMqMJ39bKQ9e9jqfLd1IMDYylPIlDe7K7zLHpnDXSzM54ZzRNDW0sOivq3jq9g+oqagPdNGUGjA0uKuAiIwOY85/TeaqB04lMiaM/3y8kz9fupQt33hhlR+llAZ3FVjHnjmKexafz4jxgyjbW8Nj1y9n2VPraGnWMfFK9YYGdxVwKWlx3Pb0dM664kgwhuVPf8O865ZRuseHq+soNcBpcFd9QkiojXN/eyw3P/FLElKj2f7dXh6+fBlrV+zQ59Mo1QM6FLIfCLbhYdXl9Sy87xM2fJ7blpY8zE76qCTSRyeT5nhNTY/z2RJ/gRRsf9/tBWvdfTEUUpfZU31ObEIkv3lsKh+98j1fLdvK7p/LKSmooqSgar/ZreGRoVagPziJtIOTyXAE/qjY8ACWXqm+QYO76pNEhBPPG8OJ540hNCSMwrwKdm0pYde2Uut1aynlhTX8/EMhP/9QuN+x7Vv5zsA/UFv5SnVGg7vq80JCbQwdkcjQEYlMmLovvbq8nvxtpezaWtIW+Au2l2krXyk0uKt+LDYhkjHZwxiTPawtraW5dV8rf6sj8HfXyj84ifSDtZWvBhYN7mpAcbuVv7WUgh3tWvmf7mvlR0SFMuygJNJGJZGSZidlmJ3kYXaSh9qJS47q1dKBSvmLBncVFDpr5e/NrSB/q3utfICwiBCSh1rBPmWYneShsW2BPyXNTmxCpAZ/1SdocFdBKyTUxrCRiQwb6bqVX7CjjJLdVZTkV1G822rh11Q0sGdnOXt2lrs8Z3hk6AGBv33LPyY+QoO/8gsN7kp14KqV71RX3UjpnmqKHd05xQVVbReAkt1V1FY1sntHGbt3lLk8d2RMmNXyHxpLSlrcAReAaHtwje9WvqPBXSkPRMWGkzbK6o93pbaqwerH373vAlDiuAAUF1RRX9NE/rZS8reVdnr+pKGxJA2JIWmInfjkaBJSo4lPjbFeU6KJiY/UG76qWxrclfKiaHsE0WMiyBiTcsA+Ywy1lQ1W4M+v3O/V+SugrrqR/K2l5G91HfzB6k6KT4km3hHsE1L2D/4JqTGOi4B2AQUzDe5K+YmIEBMfSUx8JJljXQf/6vJ6dueUUrqnmtryJsqLaqgorqWiqJby4loqimuprWygdE91tw9WCw1zXgRiSEiJJi7F+hXgDP7xjs/RcXoRGIg0uCvVR4gI9sQowqNTyBqX0ulzRhrrm62A79jKi2ooL6qloqiGipI6Khyf66obrV8Fu7u5CISHWK3+5H2/BuyJUdiToohLiiI2IbLtfWRMmF4I+gkN7kr1M+GRoaSmx5GaHtdlPudFoC34F1sXgHLHLwHnvvqaJorzqyjOr+r2u0NCbdiTorA7Ar49MZLYxAMvAs73EVGhejEIEA3uSg1Q7l4E6mub9v0ScFwIqsvrqSqro6q0jqoyx/uyehpqmygvrKG8sMatMoRFhLT9CnBeEGITI7F3ckFArwNeo8FdqSAXGR1GZGY8gzPju83bWN9MdXk9laV1VJe1C/yldVSV17ddDKrL6qgsraOpocWt+wNO4VGhxMZHEBMfSWx8JDHxEcQkRBIT50xzvCZEEh0XQWx8BFH2CB095IIGd6WU28IjQ0kaEkvSkFi38jfUNe3f+u/43nFBqHa8NtY1U1rXTOke934ZAIhNiLaHO25WR+y7KMRbF4XYBNefwyMHdpeRBnellM9ERIURkRZGSlrXXUNgjRaqKK2mtrKBxjpDTUU9NRUN1FTUU+14ralooKaygZryeqodn+uqGx35GjwqW2h4iOMXwb4LQnScY4sNt17t1ueYuAiiHGkxcRGEhPb9Rew0uCul+gQRISo2nKjYcI9WJGppbqW2ygru1eX1VvB3BP7qinpqHa8dLxRNDS1t9xo8FREVSpQ9gmi7I+A7LgJR9nCi7dYFINoeTlS7986LRVhEiF9+MWhwV0r1ayGhNuumbWKUR8c11je3BfvaSuvCUFtlva+tatz/fWWD9dmR3lDXTENds9s3ltsLDbPt96sgOjaciJhQhh+aypTZR3p8vk6/x2tnUkqpfiQ8MpTwyFgSB7t3/8DJGENDbRO1VY3UVFoXgLrqfe/bLgrVjQdeHCobaG5qpbKkjsqSuv3O29zYwpTZ3qufBnellPKAiBAZE05kTLjbN5bba6xvPuAXQkVpDfEp0V4tpwZ3pZTyI+sXQygJqTFtaQ0Nnt0Mdkffv+WrlFLKYxrclVJqANLgrpRSA5AGd6WUGoD8GtxFJEREHhGRIhGpEpGlInLgg62tvCeJyDciUioiFY73MzvJe7iINIrISt/WQCml+gd/t9znAjOA44B0R9qLneTdDJwHJAMJwM3ASyIyrn0mEQkFngM+90WBlVKqP/L3UMhrgfuNMTsAROR2YJuIZBljctpnNMYUOt+LiA1oxboYjQI2tct6J7AW2Auc6E4hjDE+GXrkK/2prN6k9Q4+wVr3fj0UUkQSgExgvTPNGLMdqASO6OK4cqABq2W+Gni/3b7xwOXAHW58/7Uisk5E1hUVFfWwFkop1T/4s+Vud7xWdEgvBzp9ZJwxJkFEIoCzgDFAM7R1x/wTuMkYU9ndg3iMMQuABQDZ2dnGkwcT9RX9sczeoPUOPsFad2/W25997s41vDquCJCA1XrvlDGmwRjzBvAL4GpH8u3AVmPMcq+WUimlBgC/tdyNMeUikgscDXwLICIjsVrtG9w8TShwsOP9FOBoESl2fI4GQh2fRxtjSr1WeKWU6mf8PVpmAXCHiIwQkTjgYWCFMWZnx4wicr6IjBeRUBGJFJFrgFOBFY4sFwKHAEc6tvlYffJHYnX1KKVU0PL3aJmHgESs0S0RwAfAbAARmQU8ZYxxPmZtqCP/UKARa2jkCpt2CgAAB2dJREFUJcaYDwCMMfvdFRWRSqDBGLOru0KsX7++WERyusvXx6QAxd3mGni03sEnWOve03pnuUoUY0zviqP8QkTWGWOyA10Of9N6B59grbu3662PH1BKqQFIg7tSSg1AGtz7jwWBLkCAaL2DT7DW3av11j53pZQagLTlrpRSA5AGd6WUGoA0uCul1ACkwb0PE5GHReRHEakUkQIReVpEkgJdLn8SEZuIfCkiRkTSuz+i/xOR00XkaxGpFpFiEXki0GXyNREZIiKvOBbyKRORj0Sk06fF9lci8isR+dzxf7rZxf4zHf/n60TkBxGZ0tPv0uDet7VgzeBNxnoscjrwfCALFAC/B2oDXQh/EZHJwGvAo1h/7+nAM4Esk588ASQBo4HBwDpguXT3uNf+pwyrrjd33OF41tbrwF+xHrD4V+DfIjK8J1+ko2X6ERE5E1hijOn0EckDiYiMBt4Fzgf+A2S483iJ/kxEvgI+NcbMDXRZ/ElENgD/cDyaGxEZA/wEpBpjBtyjCBwX8ZXGmNB2afcBpxpjTmqX9rkj332efoe23PuX04DvAl0If3CsvvUc8AeC5EFwIhIDHIv1dNNvHF0yn4hIMEzFfwQ4X0RSRSQSa9W2VQMxsHfhCNotZuTwDV0sZtQVDe79hIicD1wP3BTosvjJTcAeY8y/A10QP0rE+j95CdYKY8OwVh57x7GS2UD2BRACFALVwEzgmoCWyP/seLiYUVc0uPcD8v/bu7sQq8oojOP/R7OsoCgoQ8yykkiRsAx1zIwQK6MywUQs0IoMsYiku27CtAuNlD7IMszKqKgLg6BMsg9MIymdpgIzcSxME8wxZzTLVhfvO3g8jXlGZ85x9jw/OLjP/lzKuM4+735nLWkS8BJwW0R8Xet4Opuky4DZwKxax1JlrQ1tlkZEfUQcJI279gLqahdW58rf0lYBm0hjzWcAc4HPJfWpZWxV9gfH0czoaJzcT3KSpgOLgVsjYnWt46mSa4HzgIbcfKX1A61e0szahdW5IqIJ2AqUPwiLNtYVybnAAOCZiNgbEQcjYgkpP42sbWhVtZHUzKjUUI5zKNbJ/SQm6SHSrIkbI2JNreOporeBSznciGV8Xj8OeLVWQVXJ88B0SYNyn+BHSQ3iv6htWJ0nj6tvAmZKOjM36LmHNExRaZe2LkFSz/xM4dT8vnd+ifSzPUzSFEm9JE0BrgaWHc+1qt2sw9pnEakh+OrSGWElDU0KKSJaKJn+mJMcpDH4fbWJqmoWkJLax0Bv0iyhm/NdfZFNID1UbSQNQ20GJkXElppG1fHuBpaWvN+f/xwQET9Jmgg8RZpMsAW4o61OdZXwVEgzswLysIyZWQE5uZuZFZCTu5lZATm5m5kVkJO7mVkBObmbmRWQk7vZSS4XD3u21nFY1+Lkbt1CrjZ4MP8GZC9JzZL6H+OYablJSFuv3tWK3ex4+DdUrbsYCWyMiGZJw4HdEbGtguNaSKUQjhARBzo6QLOO5Dt36y7qSGVlIRUmq7RWT0TEjvJX68Y8ZPKCpEW5PdzvkubnSoet+5wjaVnetl/SKkmDSy8iaURuLdcsqSkv9y3ZpYekebnG+2+SFpRdY6Kk+nz+3ZI+7WYVFa2Mk7sVlqT+kvZI2gM8AszIy/OACXlbR/QnncrhCoYzSI0mStuovQIMB24nNeNoAT6QdHqO80pgNameyihgBPAWR36znkqqM1RHKoX8MDA5H38B8CapwNQVwHXAax3w97IuzLVlrLBywbF+pGYH64FhQDOwAbgF2AbsO1q3H0nTSEWemss21UdEXd7nE1JTjcsj/2eS9BjwQET0kzSQVPFwTER8lrefna89OyKWSFoOXBIRbZa3zdc4rXS7pI+Axoi4T9JVpA4+F0dEYzv+iazAPOZuhRURfwNbJd0JfBUR9ZJGATtbE20FWkhlh0v9WfZ+XRx5l7QWmCPpLNKd9D95XWtcTZK+BQblVUOBY3WcKi99ux04Py9vJDW7aJC0Mi+/ExG7jnFOKzAndyssSd8BF5FKyPaQtI/0M39KXm6MiMH/dw7SmPvmTgqxPV+b/2rj2B4AEXFI0jjScM444F7gSUljIqJb9Ny1//KYuxXZeNJd9w7grrzcQBqvLm0CcqKGq7Tgfkqy2yNiL/ADZR2F8h39EOD7vOob4IYTCSCStRHxOHAN6c5+8omc07o237lbYUVEY37Y2AdYQbrbHQy8GxG/Vnga5XOU2xURh/JyX2Bhfjg7hNQ96Ykcw4+SVgCLJd1Pang8l9QX8418/HxgnaQXgeeAA8BoYGUl0zUljQDGAh8CO0nDPBdy+MPDuiEndyu660nj7QckjQZ+aUdih9Ssua39B5JmtwAsB3oCX5I+QF4Gni7ZdzqwEHiP1F1pDXBTROwHiIgNksaSZvGsI43prwferzDGJtIsmwdJDZV/BuZExOsVHm8F5NkyZicgz2RpiIhZtY7FrJTH3M3MCsjJ3cysgDwsY2ZWQL5zNzMrICd3M7MCcnI3MysgJ3czswJycjczK6B/AdesAnDTZ/MmAAAAAElFTkSuQmCC","text/plain":["<Figure size 360x360 with 1 Axes>"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["# Plot training and validation loss\n","\n","fig, ax = plt.subplots(figsize=(5,5))\n","\n","lw = 2\n","\n","plt.tight_layout()\n","ax.plot(np.linspace(1, len(training_loss), num=len(training_loss)),training_loss, color='rebeccapurple',\n","         lw=2, linestyle='-', label='Training Loss')\n","\n","ax.plot(np.linspace(1, len(training_loss), num=len(training_loss)),validation_loss, color='r',\n","         lw=2, linestyle='-', label='Validation_loss')\n","\n","ax.set_xlabel('# Epochs',fontsize=14)\n","ax.set_ylabel('# Loss',fontsize=14)\n","ax.legend(loc=\"best\",fontsize=12)\n","\n","ax.tick_params(axis='x', labelsize=13)\n","ax.tick_params(axis='y', labelsize=13)\n","\n","\n","plt.grid(color='gray', linestyle='-', linewidth=0.1)\n","\n","\n","#plt.savefig('./loss_curves.pdf',dpi=100,bbox_inches='tight')\n","plt.show()"]}],"metadata":{"colab":{"collapsed_sections":[],"name":"Problem 1.ipynb","provenance":[]},"kernelspec":{"display_name":"Python 3.9.7 ('ml4timeseries')","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.7"},"vscode":{"interpreter":{"hash":"c93591c313a95c8a89c667763bfd86bbb342a6e2dd3e91f10558630342cde4f4"}}},"nbformat":4,"nbformat_minor":0}
